A New Method for Analyzing the Performance of the Harmony Search Algorithm

Abstract

A harmony search (HS) algorithm for solving high-dimensional multimodal optimization problems (named DIHS) was proposed in 2015 and showed good performance, in which a dynamic-dimensionality-reduction strategy is employed to maintain a high update success rate of harmony memory (HM). However, an extreme assumption was adopted in the DIHS that is not reasonable, and its analysis for the update success rate is not sufficiently accurate. In this study, we reanalyzed the update success rate of HS and now present a more valid method for analyzing the update success rate of HS. In the new analysis, take-k and take-all strategies that are employed to generate new solutions are compared to the update success rate, and the average convergence rate of algorithms is also analyzed. The experimental results demonstrate that the HS based on the take-k strategy is efficient and effective at solving some complex high-dimensional optimization problems.

1. Introduction

Harmony search (HS) [1,2,3,4,5], which is a meta-heuristic search algorithm that mimics the process of improvising a musical harmony, has been extensively employed in the fields of complex scientific computing and engineering optimization. Deng et al. developed an improved HS algorithm to construct examples for an algebra system in 2015 [6]. Tuo et al. proposed a hybrid algorithm based on HS and teaching-learning-based optimization (TLBO) for solving complex high-dimensional optimization problems; the algorithm showed efficient performance [7]. In the fields of engineering optimization, an enhanced HS was proposed to solve dynamic economic emission dispatch problems; the experimental results indicated better solutions and less computational time [8]. An adaptive dynamic HS is presented for optimizing aircraft panels [9]. Xu and Wang developed a harmony search algorithm for minimizing serve energy by examining information caching and upgrading techniques of the Internet of Things (IoT) [10]. To solve complex engineering design optimization, Yi et al. proposed an online variable-fidelity surrogate-assisted HS with a multi-level screening strategy [11]. In addition, HS has also been applied in manufacturing systems [12], microgrid planning with ESS [13], flow shop scheduling problems [14] etc. [15,16,17].

In recent years, various variants of HS were proposed to improve the performance for solving complex optimization problems [17,18,19,20,21,22,23,24], such as accelerating search speed, enhancing the power of global exploration and balancing the tradeoff between diversification and intensification. Zhang et al. reviewed the HS and its variants with respect to the algorithm structure [23]. D. Manjarres et al. analyzed the main characteristics and application portfolio of HS, and presented future research lines of HS in 2013 [24]. Nasir et al. compared the study of HS in China, Japan and Korea [25]. Although, the HS and its variants have shown strong power for solving some complex optimization problems, they have low-efficiency and low-precision solutions for solving high-dimensional (>=500) multimodal optimization problems. In 2015, an HS algorithm named “A harmony search algorithm for high-dimensional multimodal optimization problems (DIHS) [4]” was proposed. In DIHS, to determine why the standard HS has low efficiency and low precision in solving high-dimensional optimization problems, the take-one search strategy and take-all search strategy were compared to analyze the success rate of newly generated solutions for updating the worst solution in harmony memory (HM). In the comparative analysis, an extreme case of solutions for the HM is adopted assuming that all solutions need only adjust one-dimensional values to achieve the global optimal solution. The analytical results [4] are as follows:

(1)

The update success rate (defined as the rate that the new generated harmony is better than the worst harmony in HM.) of the take-one strategy is

$$R_{take-one}=\frac{1}{D}\times \frac{HMS-1}{HMS}$$

(1)

where D is the dimension of the optimization problem and HMS denotes the size of the harmony memory.

(2)

The update success rate of the take-all strategy is

$$R_{take-all}={\left(\frac{HMS-1}{HMS}\right)}^D$$

(2)

In recent research, we identified a key analysis error in the article of DIHS, when the dimension D is larger than the HMS, the update success rate formulas [4] are incorrect. Although the error does not affect the correctness of the final conclusion of the article, it may cause readers to misunderstand and incorrectly apply the DIHS algorithm. To correct the error, in this work, we introduce a new method for analyzing the take-one search and take-all search strategies.

The contributions of this article are listed as follows:

(1)

New analysis strategies, namely, the take-k and take-all schemes, are presented. The analysis assumptions used are closer to the conditions in the actual optimization process than are the previous extreme assumptions.

(2)

An adaptive take-k strategy is proposed for improving the harmony search algorithm.

The remainder of this paper is organized as follows. HS is introduced in Section 2. Section 3 presents detailed analyses of the take-one strategy and take-all strategy. Section 4 introduces new take-k and take-all strategies. The simulation experiments are performed and the results are analyzed in Section 5. In Section 6, discussions are presented.

2. Standard Harmony Search Algorithm

HS is a very simple real-code optimization algorithm that can be easily used to solve continuous and discrete optimization problems. This algorithm contains three operators:

(1) The combinatorial operator is designed to select values from HM and obtain a new harmony. The objective is to improve the new harmony based on the historical memory of musicians.

(2) The local-adjusting operator fine-tunes the potential solutions near the new harmony with the aim of finding the most precise solution. This approach can aggravate spatial disturbances associated with a large fret width and fine-tune harmonies with a small fret width.

(3) The random search operator is designed to explore unknown search areas and prevent the algorithm from falling to local optima.

The pseudocode of the standard HS algorithm is expressed as algorithm 1:

Algorithm 1. The pseudocode of standard HS algorithm.

Input: MaxT: maximum number of iterations HMCR: harmony memory consideration rate HMS: harmony memory size PAR: pitch-adjusting rate fw: fret width Output: Best harmony Xbest

Randomly initialize HM               $\mathrm{HM}=\left[\begin{array}{c}{X}^1\\ X^2\\ ⋮\\ X^{HMS}\end{array}\right]=\left[\begin{array}{cccc}{x}_1^1& x_2^1& \dots & x_D^1\\ x_1^2& x_2^2& \cdots & x_D^2\\ ⋮& ⋮& ⋮& ⋮\\ x_1^{HMS}& x_2^{HMS}& \cdots & x_D^{HSM}\end{array}\right]$      $x_i^j=x_i^L+rand(0,1)\times \left(x_i^U-x_i^L\right)$, i = 1, 2…, D; j = 1, 2…, HMS.      $x_i^L$ and $x_i^U$ denote the lower bound and upper bound of the ith decision variable, respectively. Improvise a new harmony $X^{new}=(x_1^{new},x_2^{new},\cdots ,x_D^{new})$ as follow. For I = 1➔D If rand(0,1) < HMCR      $x_i^{new}=r\in \left\{x_i^1,x_i^2,\cdots ,x_i^{HMS}\right\}$      If rand(0,1) < PAR      $x_i^{new}=x_i^{new}+rand(0,1)\times f{w}_i$      End Else      $x_i^{new}=x_i^L+rand(0,1)\times \left(x_i^U-x_i^L\right)$ End End Update HM with the new harmony as follows:     $X^{IDworst}=\{\begin{array}{c}{X}^{new},\\ X^{IDworst},\end{array}\begin{array}{c}{X}^{new}\mathrm{is}\mathrm{better}\mathrm{than}{X}^{IDworst}\\ otherwise\end{array}$ where IDworst is the index of the worst harmony in HM. If the terminal condition is met, then go to step 2. Output the best harmony of HM.

3. Take-One Strategy and Take-All Strategy

In the original paper about DIHS, an extreme case of solutions in the HM is adopted assuming that all solutions need only adjust one-dimensional values to achieve the global optimal solution. HM can be converted to the following matrix by swapping rows and columns.

$$\mathrm{HM}=\left[\begin{array}{cccc}{x}_1^1& x_2^{*}& \cdots & x_{HMS}^{*}\\ x_1^{*}& x_2^2& \cdots & x_{HMS}^{*}\\ ⋮& ⋮& ⋮& ⋮\\ x_1^{*}& x_2^{*}& \cdots & x_{HMS}^{HMS}\end{array}\begin{array}{c}{x}_{HMS+1}^{*}\\ x_{HMS+1}^{*}\\ ⋮\\ x_{HMS+1}^{*}\end{array}\begin{array}{c}\cdots \\ \cdots \\ ⋮\\ \cdots \end{array}\begin{array}{c}{x}_D^{*}\\ x_D^{*}\\ ⋮\\ x_D^{*}\end{array}\right]$$

(3)

In the HM, $x_i^{*}\left(i=1,2,\dots ,D\right)$ indicates that the ith decision variable has an optimal value. $x_1^1,x_2^2\cdots ,x_{HMS}^{HMS}$ are decision variables for which the optimal value has not been obtained.

Under this assumption,

(1) The success rate of update of the take-one strategy should be $\frac{HMS}{D}\ast \frac{HMS-1}{HMS}$, not $\frac{1}{D}\ast \frac{HMS-1}{HMS}$, where $\frac{HMS}{D}$ indicates the probability of selecting one of the previous HMS columns from HM to be adjusted. $\frac{HMS-1}{HMS}$ represents the probability of obtaining the best value $x_i^{*}\left(i=1,2,\dots ,D\right)$ of the ith decision variable.

(2) The success rate of update of the take-all strategy should be ${\left(\frac{HMS-1}{HMS}\right)}^{HMS}$, not ${\left(\frac{HMS-1}{HMS}\right)}^D$, because all of the values of decision variables $x_{HMS+1},x_{HMS+2},\cdots ,x_D$ are optimal.

Under this condition, the update success rate of the take-all strategy is independent of the dimension of the problem, which is inconsistent with the results of the previous analysis. If we assume that there is one and only one nonoptimal value in all columns of the matrix HM, formulas (1) and (2) are correct.

However, based on many experimental results, the concept of the DIHS algorithm should be correct, but the premise assumptions were too extreme for this analysis. Therefore, in this work, we employ more realistic hypothetical conditions to reanalyze the dynamic-dimensionality-reduction adjustment (DDRA) strategy using take-k strategy and take-all strategy.

4. Take-K Strategy and Take-All Strategy

4.1. Concepts and Assumptions

The take-k strategy means that k decision variables of ${\mathit{X}}^{new}(={\mathit{X}}^{IDworst})$ are chosen for adjusting. If the adjusted ${\mathit{X}}^{new}$ is better than the old ${\mathit{X}}^{IDworst}$, then the ${\mathit{X}}^{IDworst}$ in HM is replaced by the adjusted ${\mathit{X}}^{new}$. If the value of k is equal to dimension D, the take-k strategy is referred to as the take-all strategy.

The take-k strategy employs the following step to improvise the harmony (see Algorithm 2).

Algorithm 2. The Pseudocode for improvising new harmony for the take-k strategy.

${\mathit{X}}^{new}={\mathit{X}}^{IDworst}$

For i = 1➔D

If rand(0,1) < k/D // each dimension has a k/D chance of being adjusted.

If rand(0,1) < HMCR

$x_i^{new}=x_i^j$ //combinatorial operator

If rand(0,1) < PAR

$x_i^{new}=x_i^{new}+rand(0,1)\times fw(i)$ //local-adjusting operator

End

Else

$x_i^{new}=x_i^L+rand(0,1)\times \left(x_i^U-x_i^L\right)$ // random search operator

End

End

End

Assumption 1.

Suppose that in the ith column of HM, there is only one value that is not equal to $x_i^{*}\left(i=1,2,\cdots ,D\right)$. In the j^th row of HM, there are D/HMS values that do not reach the optimal value, as follows:

$$\mathrm{HM}=\left[\begin{array}{cccc}{y}_1& x_2^{*}& \cdots & x_{HMS}^{*}\\ x_1^{*}& y_2& \cdots & x_{HMS}^{*}\\ ⋮& ⋮& ⋮& ⋮\\ x_1^{*}& x_2^{*}& \cdots & y_{HMS}\end{array}\begin{array}{c}{y}_{HMS+1}\\ x_{HMS+1}^{*}\\ ⋮\\ x_{HMS+1}^{*}\end{array}\begin{array}{c}{x}_{HMS+2}^{*}\\ y_{HMS+1}\\ ⋮\\ x_{HMS+2}^{*}\end{array}\cdots \begin{array}{c}\cdots \\ \cdots \\ ⋮\\ \cdots \end{array}\begin{array}{c}{x}_D^{*}\\ x_D^{*}\\ ⋮\\ y_D\end{array}\right]$$

(4)

Assumption 2.

Suppose that in generation t, the probability that each dimension in HM is able to obtain a better value by employing three optimal operators (combinatorial operator, local-adjusting operator, and random search operator) than it currently holds is equal to$p(t)=\frac{HMS-1}{HMS}$.

In the take-k strategy, first ${\mathit{X}}^{new}={\mathit{X}}^{IDworst}$; then, each dimension $x_i^{new}$ (i = 1,2,…,D) has a k/D chance of being updated by the three operators. If k==D, the take-k and take-all strategies are equivalent.

If $x_i^{new}$ is chosen from HM for adjustment, the probability of improving $x_i^{new}$ is p(t), and the probability of the value worsening is 1 − p(t).

Assumption 3.

Suppose that there are k decision variables in ${\mathit{X}}^{new}$ that are selected to be adjusted. If more than 1/2 of the decision variables achieved improved values, a better solution was obtained.

Definition 1.

Gain value.${\mathit{X}}^{new}(t)$ is a new improvised solution in the t-th iteration. The gain value $G(t)$ obtained at the t-th iteration is defined as

$$G(t)=\{\begin{array}{c}f\left(X^{IDworst}\right)-f\left(X^{new}\right)\\ 0\end{array}\begin{array}{c},\\ ,\end{array}\begin{array}{c}f\left(X^{new}\right)<f\left(X^{IDworst}\right)\\ otherwise\end{array}$$

(5)

The gain value $G(\Delta t)$ after $\Delta t$ iterations is defined as

$$G(\Delta t)=f\left(X^{IDbest}\left(t_2\right)\right)-f\left(X^{IDbest}\left(t_1\right)\right)$$

(6)

where $\Delta t=t_2-t_1$, $X^{IDbest}\left(t_2\right)$ and $X^{IDbest}\left(t_1\right)$ denote the best solutions at times $t_2$ and $t_1$ ($t_2>t_1$), respectively.

4.2. Update Success Rate of Take-k and Take-All Strategies

To analyze the update success rate of the take-k and take-all strategies, Figure 1 introduces the judging criteria for generating a better solution, ${\mathit{X}}^{new}$. In the take-k strategy, first, the values of ${\mathit{X}}^{new}$ are initialized to the values of the worst harmony ${\mathit{X}}^{IDworst}$ in HM. Second, ${\mathit{X}}^{new}$ is adjusted according to algorithm 2. If more than 1/2 of the decision variables of ${\mathit{X}}^{new}$ have been improved, we think ${\mathit{X}}^{new}$ is improved successfully. Otherwise, ${\mathit{X}}^{new}$ is considered to be an improvision.

By assumption 3, the probability that a better solution can be obtained using the take-k strategy is

$$R_{take-k}={\displaystyle \sum _{i=\raisebox{1ex}{$k$}\!\left/ \!\raisebox{-1ex}{$2$}\right.+1}^k{C}_k^i\times p{(t)}^i{(1-p(t))}^{k-i}}$$

(7)

where i = k/2 + 1 to k, which means that among the k decision variables, more than k/2 decision variables obtain better values.

The take-all strategy is equivalent to the take-D strategy (k = D), and the probability of obtaining a better solution is

$$R_{take-all}={\displaystyle \sum _{i=\raisebox{1ex}{$D$}\!\left/ \!\raisebox{-1ex}{$2$}\right.+1}^D{C}_D^i\times p{(t)}^i{(1-p(t))}^{D-i}}$$

(8)

From Equation (7), the update success rate $R_{take-k}$ is independent of the dimension D. Figure 2 shows the success rate curves of the take-k (k = 0.1D) and take-all strategies as the dimension D changes.

As shown in Figure 2, when p(t) < 0.5, the update success rate for both strategies decreases with dimension D, but the take-all curve decreases much faster than the take-k curve. When p > 0.5, the update success rate of the take-all strategy is higher than that of the take-k strategy for a high-dimensional optimization problem; moreover, if D > 500, the success rate of the take-all approach is close to 1. However, in practice, the value of p(t) is usually less than 0.5.

Figure 3 shows a comparison of the update success rate of the take-k and take-all strategies when using p(t) ∈ {0.3, 0.25, 0.2, 0.15, 0.1, 0.05}. In Figure 3a,b, the x-axis represents the probability that decision variable $x_i\left(i=1,2,\dots ,D\right)$ in HM can obtain a better value than the current value; the y-axis shows the update success rate. We can see that as D and k increase, the success rate of updating the HM significantly decreases. Moreover, the smaller the value of p(t) is, the smaller the success rate.

Generally, for a complex optimization problem, in the search process of an intelligent optimization algorithm, the population has excellent initial diversity, and each decision variable has a high probability of achieving an improved value. As the search progresses, the diversity of the population decreases, and the probability of finding better individuals decreases. However, if the population is clustered around a local optimal solution (the local optimal solution has not yet been obtained), then the probability of finding a better solution is usually high. However, if the population has converged to a local optimum (the local optimal solution has been found), the probability of finding a better solution is 0.

To adapt to the search process, DIHS adopted a DDRA strategy. An adaptive take-k strategy can be employed in the DIHS algorithm as follows:

$$k=K_{\max }-(K_{\max }-K_{\min })\cdot {\left(\frac{t}{MaxFEs}\right)}^a$$

(9)

where $K_{\max }$ and $K_{\min }$ represent the maximum and minimum values of k, respectively. a is a parameter used to adjust the rate of descent of k, taking a value of 2 in this paper. MaxFEs denotes the Maximum number of function evaluations (FEs).

The parameters PAR and fw are the same as those in the original DIHS method [4].

$$\mathrm{PAR}(t)={\mathrm{PAR}}_{\min }+({\mathrm{PAR}}_{\max }-{\mathrm{PAR}}_{\min })\times \frac{t}{\mathrm{MaxFEs}}$$

(10)

$$fw(t)=\{\begin{array}{c}f{w}_{\max }\times {\left(\frac{f{w}_{\mathrm{mid}}}{f{w}_{\max }}\right)}^{{\left(\frac{t}{\mathrm{MaxFEs}/2}\right)}^2}\begin{array}{cc},& t\le \frac{\mathrm{MaxFEs}}{2}\end{array}\\ f{w}_{\mathrm{mid}}\times {\left(\frac{f{w}_{\min }}{f{w}_{\mathrm{mid}}}\right)}^{{\left(\frac{t-\mathrm{MaxFEs}/2}{\mathrm{MaxFEs}/2}\right)}^2}\begin{array}{cc},& t>\frac{\mathrm{MaxFEs}}{2}\end{array}\end{array}$$

(11)

where ${\mathrm{PAR}}_{\max }$ and ${\mathrm{PAR}}_{\min }$ represent the maximum and minimum pitch adjustment rates, respectively; $f{w}_{\min }$ and $f{w}_{\max }$ denote the minimum and maximum fret widths, respectively; and $f{w}_{mid}$ is a value between $f{w}_{\min }$ and $f{w}_{\max }$.

The parameters PAR and fw are both used to balance the exploration power and exploitation power of DIHS. Equation (9) is equivalent to Equation (1) of literature [4] in which the parameters a, HMS, PAR and fw have been analyzed in detail.

5. Simulation Experiments

5.1. Experimental Environment and Parameter Settings

To investigate the take-k and take-all strategies, six classical benchmark functions [26] (see

Table 1. Six classical benchmark functions.

Function Name and Formula	Function Graph
Ackley$f(X)=-20e^{-\frac{1}{5}\sqrt{\frac{1}{D}{\displaystyle \sum _{i=1}^D{x}_i^2}}}-e^{\frac{1}{D}{\displaystyle \sum _{i=1}^D\cos (2\pi x_i)}}+20+e$
Griewank$f(X)=\frac{1}{4000}{\displaystyle \sum _{i=1}^D{x}_i^2}-{\displaystyle \prod _{i=1}^D\left(\cos (\frac{x_i}{\sqrt{i}})\right)}+1$
Levy$\begin{array}{l}f(\mathit{X})={\sin }^2(\pi y_1)+{\displaystyle \sum _{i=1}^{D-1}\left[{(y_i-1)}^2\left(1+10{\sin }^2(\pi y_i+1)\right)\right]}\\ +{(y_D-1)}^2\left(1+10{\sin }^2(2\pi y_D)\right)\hfill \end{array}$ $y_i=1+\frac{x_i-1}{4},i=1,2,\cdots ,D$
Michalewics$f(\mathit{X})=-{\displaystyle \sum _{i=1}^D\sin (x_i){\left(\sin (i{x}_i{}^2/\pi )\right)}^{2m},m=10}$
Rastrigin$f(\mathit{X})=10D+{\displaystyle \sum _{i=1}^D\left(x_i{}^2-10\cos (2\pi x_i)\right)}$
Schwefel 2.26$f(\mathit{X})=418.982887272434D-{\displaystyle \sum _{i=1}^D{x}_i^2\sin (\sqrt{\left|x_i\right|})}$

) are employed to test the performance of the two strategies. Take-all-based HS, take-k-based HS and take-k-based DIHS are compared based on the six benchmark functions. The dimensions (D) of all functions are equal to 1000 in the experiments. In the take-k-based HS, the value of k is set to 0.01D. In the take-k-based DIHS, parameters are set as follows:

$$K_{max}=0.1D,K_{min}=0.01D;PA{R}_{max}=0.75,PA{R}_{min}=0.02;$$

$$f{w}_{\max }=(x_i^U-x_i^U)/500;f{w}_{\min }=(x_i^U-x_i^U)\times {10}^{-15};f{w}_{mid}=(x_i^U-x_i^U)\times {10}^{-5}.$$

Simulation experiments are performed to compare the update success rate and convergence of the three algorithms. Figure 4 shows the success rate curves and convergence curves. To further investigate the search ability of the three algorithms, we trace the values of the gains obtained during time $\Delta t$ and the convergence rate.

5.2. Update Success Rate and Convergence Speed

As shown in Figure 4, at the beginning of the search process, the take-all-based HS has a higher update success rate than the take-k-based HS. As the search progresses, the update success rate of the take-all-based HS decreases dramatically, and the take-k-based HS and DIHS methods are able to maintain a high update success rate during the search. The take-all-based HS approach can be compared to take-k-based DIHS; the former has a higher success rate at the beginning stage of the search. However, in the mid to late stages of the search, the update success rate for the take-k-based DIHS gradually increases and exceeds the success rate for the take-k-based HS.

From the convergence curves of the algorithms in Figure 5, the take-all-based HS algorithm has the slowest convergence rate among the three compared algorithms. The take-k-based HS has a high convergence rate during the initial stage of the search, but it is much slower than take-k-based DIHS overall. Of the three algorithms, DIHS has the strongest ability to obtain high-precision optimal solutions for all test functions.

Figure 6 presents the gain values that are traced during the search. Although the take-k strategy has a higher probability of finding an improved solution than the take-all strategy during the search process, the improvement over the best solution $G(\Delta t)$ of the take-k strategy may be small. As shown in Figure 6, the standard HS algorithms that are based on the take-k and take-all strategies have higher gain values than the take-k-based DIHS at certain times, but they are prone to have zero (less than 1 × 10⁻³⁰) gain values at points in the search process. The take-k-based DIHS seldom obtains a zero-gain value before the global optimal solution is reached, except for the Michalewics function. The gain values of the take-k-based DIHS become very small late in the search process because the DIHS algorithm has converged to the global optimal solution.

To further investigate the performance of the take-k strategy for HS, six classical high-dimensional optimization problems with multimodal functions (sphere shift function, Schwefel shift function, Rastrigin shift function, Ackley shift function, Griewank shift function and fast fractal double dip function) [27,28] are tested. Figure 7 presents the update success rate of three algorithms (take-k-based HS, take-all-based HS and take-k-based DIHS); the results confirm the previous conclusions.

Table 2. Comparisons of six algorithms for six high-dimensional optimization problems (D=1000) (Best/Mean/Worst denotes the best/mean/worst fitness values, respectively, obtained in 30 runs).

Algorithm	Function	Best	Mean	Worst	Runtime	Function	Best	Mean	Worst	Runtime
					(s)					(s)
HS (Take-all)	Sphere shift	6.13 × 104	6.44 × 104	6.69 × 104	1.02 × 103	Rastrigin shift	1.31 × 103	1.35 × 103	1.38 × 103	1.52 × 103
Take-k based HS		1.28 × 10−1	1.32 × 10−1	1.36 × 10−1	3.77 × 102		9.76 × 10−2	1.07 × 10−1	1.26 × 10−1	6.14 × 102
SaDE		4.05 × 10−3	8.70 × 10−3	1.34 × 10−2	1.08 × 103		5.79 × 103	6.97 × 103	8.16 × 103	2.36 × 103
CoDE		2.25 × 10−3	4.75 × 10−3	7.25 × 10−3	5.38 × 102		3.59 × 103	3.81 × 103	4.04 × 103	1.08 × 103
CLPSO		5.14 × 10−1	5.51 × 10−1	5.87 × 10−1	3.81 × 103		4.20 × 102	4.56 × 102	4.92 × 102	4.27 × 103
Take-k based DIHS		5.90 × 10−34	1.01 × 10−32	2.43 × 10−32	4.85 × 102		0	0	0	1.26 × 103
HS (Take-all)	Schwefel shift	5.75 × 101	5.81 × 101	5.87 × 101	1.32 × 103	Griewank shift	5.64 × 102	5.77 × 102	5.91 × 102	1.60 × 103
Take-k based HS		4.85 × 101	5.13 × 101	5.43 × 101	3.70 × 102		7.32 × 10−3	3.23 × 10−2	7.10 × 10−2	8.08 × 102
SaDE		8.04 × 101	8.14 × 101	8.23 × 101	2.15 × 103		1.19 × 10−3	4.11 × 10−2	8.09 × 10−2	1.83 × 103
CoDE		1.15 × 102	1.16 × 102	1.17 × 102	6.57 × 102		3.32 × 10−4	9.16	1.83 × 101	7.50 × 102
CLPSO		6.99 × 101	7.48 × 101	7.97 × 101	4.30 × 103		2.77 × 10−2	2.94 × 10−2	3.12 × 10−2	4.38 × 103
Take-k based DIHS		2.56 × 101	2.60 × 101	2.64 × 101	6.39 × 102		2.44 × 10−15	2.50 × 10−15	2.55 × 10−15	8.66 × 102
HS (Take-all)	Rosenbrock shift	2.21 × 109	2.37 × 109	2.48 × 109	1.58 × 103	Ackley shift	9.23	9.4	9.62	1.65 × 103
Take-k based HS		2.10 × 103	2.28 × 103	2.62 × 103	7.99 × 102		1.91 × 10−2	1.97 × 10−2	2.00 × 10−2	9.71 × 102
SaDE		4.03 × 103	4.42 × 103	4.81 × 103	1.75 × 103		1.58 × 101	1.59 × 101	1.60 × 101	1.86 × 103
CoDE		2.84 × 103	2.89 × 103	2.94 × 103	6.18 × 102		1.46 × 101	1.49 × 101	1.53 × 101	7.80 × 102
CLPSO		5.94 × 103	6.26 × 103	6.58 × 103	4.12 × 103		1.35 × 101	1.54 × 101	1.73 × 101	4.43 × 103
Take-k based DIHS		1.15 × 103	1.15 × 103	1.16 × 103	7.28 × 102		2.31 × 10−13	2.31 × 10−13	2.31 × 10−13	9.82 × 102

summarized the test results of six algorithms (take-all-based HS, take-k-based HS, SaDE [29], CoDE [30], CLPSO [31] and take-k-based DIHS) for the six high-dimensional optimization problems.

As shown in Table 2, the best/mean/worst solution of take-k-based DIHS is superior to those of the other algorithms for all functions. The take-k-based HS (k = 0.01D) has the shortest run time. Additionally, take-all-based HS requires more than twice the run time of the take-k-based HS algorithm. The take-k-based DIHS requires a slightly higher run time than the take-k-based HS algorithm. Compared to SaDE, CoDE and CLPSO, the take-k-based DIHS is optimal for all metrics.

5.3. Analysis of the Take-k-Based DIHS Results

As shown in Figure 4 and Figure 7, for most test functions, the update success rate curve of the DIHS algorithm displays an “N-shaped” trajectory. The initial value is close to 50%, and this value then rapidly decreases to between 0.1% and 1%. Next, the curve slowly rises (to approximately 35%), and in the final stage, the success rate suddenly and rapidly drops to a very low value (close to 0).

We believe that this phenomenon is due to the following factors.

(1)

The population is randomly initialized, and every individual has a bad fitness value. In this case, DIHS has a higher probability than the other algorithms of finding an improved solution.

(2)

In the early stage of the search, the DIHS algorithm perturbs the search space to find the global optimal region with a high perturbation step fw, resulting in a rapid decrease in the probability of finding a better solution than the current solution.

5.4. Convergence Rate

To test the convergence ability of algorithms during the search, we employ the average convergence rate of HS for t generations. The average convergence rate at time t [32] is defined as Formula (12):

$$\mathrm{R}(\mathrm{t}|\mathrm{H}{\mathrm{M}}_0)=1-{\left(\left|\frac{{\mathrm{f}}^{\mathrm{o}\mathrm{p}\mathrm{t}}-{\mathrm{f}}^{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}\left(\mathrm{H}{\mathrm{M}}_{\mathrm{t}}\right)}{{\mathrm{f}}^{\mathrm{o}\mathrm{p}\mathrm{t}}-{\mathrm{f}}^{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}\left(\mathrm{H}{\mathrm{M}}_0\right)}\right|\right)}^{1/\mathrm{t}}$$

(12)

where $\mathrm{H}{\mathrm{M}}_0$ and $\mathrm{H}{\mathrm{M}}_{\mathrm{t}}$ respectively denote the initial population (harmony memory, HM) and the harmony memory after t generations. ${\mathrm{f}}^{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}\left(\mathrm{H}{\mathrm{M}}_{\mathrm{t}}\right)$ represents the best fitness value among the individuals of $\mathrm{H}{\mathrm{M}}_{\mathrm{t}}$. ${\mathrm{f}}^{\mathrm{o}\mathrm{p}\mathrm{t}}$ denotes the optimal fitness of the problem.

Figure 8 shows the convergence rates of 8 complex benchmark functions by using standard HS (take-all-based HS), take-k-based HS and take-k-based DIHS algorithms. It can be seen from the figure that on all functions, the take-k-based method is superior to the take-all method, and the take-k-based DIHS has a higher convergence rate than other algorithms. At the beginning of the search process, the take-k-based DIHS can maintain a high convergence rate that declines slowly. In the second half of the search, the take-k-based DIHS has a progressively higher convergence rate for most functions (e.g., Ackley, Griewank and so on). In particular, the convergence rate on the Rastrigin shift function grows to 1. The results of this experiment further validate that the take-k strategy and dynamic dimensionality reduction are correct and effective improvement strategies for improving harmony search to solve complex high-dimensional optimization problems.

6. Discussion

Although many algorithms, such as cooperative coevolution (CC) algorithms [33,34,35,36] (decomposition-based algorithms), have been designed to solve high-dimensional optimization problems and achieved notable progress, they are still ineffective for some complex optimization problems [37]. This work aims to discover why effective HS algorithms that are able to solve small-scale optimization problems are not effective in solving high-dimensional optimization problems. By comparing take-k-based HS with take-all-based HS, it is found that take-all-based HS has a very low update success rate in the later search stage when solving high-dimensional optimization problems.

At the beginning of the search, a large k-value in the take-k-based HS is beneficial for accelerating the exploration process and obtaining a large gain value $G(\Delta t)$ at each iteration, but the update success rate decreases rapidly as the search progresses. A small k-value is beneficial for achieving a high update success rate, but the obtained gain value at each iteration is small.

To improve the performance of take-k-based HS, DIHS employs the dynamic take-k strategy, dynamic pitch adjustment rate and dynamic fret width. The experimental results demonstrate that the dynamic take-k strategy is very effective for balancing the update success rate and gain value in the search process. However, the take-k-based DIHS is still not effective for obtaining high-precision globally optimal solution for some high-dimensional optimization problems (e.g., Rosenbrock function) in which all decision variables are highly interconnected.

In this work, the goal is to improve the HS algorithm without employing any complex strategies, such as CC and decomposition-based algorithms. The proposed approach can be used in various applications and requires a search algorithm that is as simple as possible. Recently, the HS algorithm and its variants have received considerable attention, such as in portfolio selection [38] and searching for pathogenic sites based on the entire human genome [39,40,41]. For HS, improving the search speed and solving complex high-dimensional optimization problems remain an important focus of future research. The take-k-based strategy is a promising approach for improving the global search power, enhancing the solution quality and accelerating the speed.