Heterogeneous Cooperative Bare-Bones Particle Swarm Optimization with Jump for High-Dimensional Problems

Abstract

This paper proposes a novel Bare-Bones Particle Swarm Optimization (BBPSO) algorithm for solving high-dimensional problems. BBPSO is a variant of Particle Swarm Optimization (PSO) and is based on a Gaussian distribution. The BBPSO algorithm does not consider the selection of controllable parameters for PSO and is a simple but powerful optimization method. This algorithm, however, is vulnerable to high-dimensional problems, i.e., it easily becomes stuck at local optima and is subject to the “two steps forward, one step backward” phenomenon. This study improves its performance for high-dimensional problems by combining heterogeneous cooperation based on the exchange of information between particles to overcome the “two steps forward, one step backward” phenomenon and a jumping strategy to avoid local optima. The CEC 2010 Special Session on Large-Scale Global Optimization (LSGO) identified 20 benchmark problems that provide convenience and flexibility for comparing various optimization algorithms specifically designed for LSGO. Simulations are performed using these benchmark problems to verify the performance of the proposed optimizer by comparing the results of other variants of the PSO algorithm.

1. Introduction

Many optimization problems in modern engineering, e.g., optimal design and scheduling problems, must be solved with finite resources that should be used efficiently. In particular, most of these problems are high-dimensional and complex [1,2,3]. Therefore, the recent focus of optimization techniques has been on solving complex and high-dimensional problems, as described in [4,5,6,7,8].

The Particle Swarm Optimization (PSO) algorithm [9,10] is a metaheuristic inspired by the social behavior of birds flocking or fish schooling; the algorithm was created by simplifying this social behavior. In the algorithm, the so-called particles find a population of candidate solutions to a given optimization problem by moving in a search space according to simple mathematical formulas that are related to the particles’ positions and velocities. The PSO algorithm can search the solution spaces of optimization problems with few or no assumptions about the problems, even those that involve searching relatively large spaces. Additionally, to be solved using the PSO algorithm, a problem need not be differentiable, and PSO can be robustly used with problems that include uncertainties such as noise or changes over time. Therefore, PSO algorithms are widely and frequently used to solve optimization problems because they are simple to implement, stable, and high-performing.

There have been many studies of PSO, which has great utility. Among these studies, there are several investigations into improving the convergence of the PSO algorithm and selecting the optimal PSO parameters [11,12,13,14]. Generally, the standard PSO algorithm has four primary user-controlled parameters. The selection of these parameters is known to have a considerable influence on the optimization algorithm’s performance. PSO typically uses a uniform distribution to generate random numbers. However, the authors of [15,16] have demonstrated that escaping local optima is difficult for this PSO algorithm. Therefore, there have been many studies on PSO variants that use Gaussian distributions [16,17,18], which can reduce or remove some of the parameters. Of these algorithms, the Bare-Bones PSO (BBPSO) algorithm [19] is the simplest, and it can be intuitively understood and easily implemented without considering the parameter settings by sampling new particle positions from a Gaussian distribution whose mean is given by the average of the globally and locally best positions and whose standard deviation is given by the distance between the globally and locally best positions. Therefore, the BBPSO algorithm has attracted much interest from researchers. This algorithm, however, can become stuck at local optima when solving high-dimensional problems with many local optima, as mentioned in [17], and is subject to the “two steps forward, one step backward“ phenomenon. This paper attempts to solve these problems and proposes an effective BBPSO-based optimizer for high-dimensional problems that combines heterogeneous cooperation based on the exchange of information between particles and a jumping strategy to avoid local optima. Figure 1 shows a brief summary of the goal and main idea for the development of the proposed Heterogeneous Cooperative BBPSO with Jumping (HCBBPSO-Jx) algorithms.

The remainder of this paper is organized as follows: In Section 2, detailed mathematical models of the standard PSO and BBPSO algorithms are briefly introduced, and the cooperative learning and jumping strategies are described. Section 3 provides a detailed explanation of the proposed HCBBPSO-Jx algorithms. In Section 4, we verify the performance of the HCBBPSO-Jx algorithms using the 20 benchmark functions provided by the IEEE Congress on Evolutionary Computation 2010 (CEC 2010) Special Session on Large-Scale Global Optimization (LSGO) to provide convenience and flexibility for comparing various optimization algorithms. Using these large-scale global optimization problems, we compare the results with those of the other PSO variants. Finally, Section 5 concludes this paper.

2. Background Knowledge

2.1. The Particle Swarm Optimization (PSO) Algorithm

This section describes the mathematical model and the search procedure of the standard PSO algorithm. First, let $f:{\mathbb{R}}^n\to \mathbb{R}$ be the cost or objective function that we should minimize. A candidate solution takes the form of a vector of real numbers, and the output of the function is a real number, which is the value of the objective function for the given candidate solution. The goal of this optimization problem is to find a solution ${\mathbf{x}}^{*}$ such that $f\left({\mathbf{x}}^{*}\right)\le f\left(\mathbf{x}\right)$ for all $\mathbf{x}$ in the search space, which is bounded by the values ${\mathbf{b}}_l$ and ${\mathbf{b}}_u$.

Algorithm 1 shows the process of the standard PSO algorithm [9,10]. The parameter $N_P$ is the size of the population (called a swarm in the PSO algorithm) or the number of particles. As mentioned above, the parameters w, ${\phi }_p$, and ${\phi }_g$ represent the inertia weight and the acceleration constants, respectively. Each particle has a position ${\overrightarrow{x}}_i\left(k\right)\in {\mathbb{R}}^n$ in the search space and a velocity ${\overrightarrow{v}}_i\left(k\right)\in {\mathbb{R}}^n$, at time k. The vector ${\overrightarrow{p}}_i\left(k\right)$ is the best known position of particle i, and the vector $\overrightarrow{g}\left(k\right)$ is the best known position of the entire swarm at time k. These are also called “$p_{best}$” and “$g_{best}$,” respectively.

Before the algorithm begins, each particle’s position and velocity are initialized with uniformly distributed random vectors, i.e., ${\overrightarrow{x}}_i\left(1\right)\sim U\left({\mathbf{b}}_l,{\mathbf{b}}_u\right)$ and ${\overrightarrow{v}}_i\left(1\right)\sim U\left(-\left|{\mathbf{b}}_u-{\mathbf{b}}_l\right|,\left|{\mathbf{b}}_u-{\mathbf{b}}_l\right|\right)$, respectively. The vector ${\overrightarrow{p}}_i\left(1\right)$ is first initialized with the vector ${\overrightarrow{x}}_i\left(1\right)$. Finally, the vector $\overrightarrow{g}\left(1\right)$ is initialized with the best of the vectors $\mathbf{p}\left(1\right)$.

Algorithm 1 Standard PSO Algorithm.
1:$SetParameters(N_P,w,{\phi }_p,{\phi }_g);$
	▹$N_P$: # of Particles
2:$Initialize\left(\mathbf{x}\right(1),\mathbf{p}(1),\overrightarrow{g}(1),\mathbf{v}(1\left)\right);$
3:$k=1;$
4: while   Termination condition does not meet do
5:      for   $i=1$ to   $N_P$ do
6:           for   $d=1$ to   n do	▹n: Dimension of Problem
7:                $r_p,r_g\sim U(0,1);$
8:                $k=k+1;$
9:                $Update\left(v_{i,d}\left(k\right)\right);$	▹Using Equation (1)
10:           end for
11:           $Update\left({\overrightarrow{x}}_i\left(k\right)\right);$
	▹Using Equation (2)
12:           if   $f\left({\overrightarrow{x}}_i\left(k\right)\right)<f\left({\overrightarrow{p}}_i\left(k\right)\right)$ then
13:                ${\overrightarrow{p}}_i\left(k\right)\leftarrow {\overrightarrow{x}}_i\left(k\right);$	▹Update $p_{best}$
14:                if   $f\left({\overrightarrow{p}}_i\left(k\right)\right)<f\left(\overrightarrow{g}\left(k\right)\right)$ then
15:                     $\overrightarrow{g}\left(k\right)\leftarrow {\overrightarrow{p}}_i\left(k\right);$	▹Update $g_{best}$
16:                end if
17:           end if
18:      end for
19:end while
20: return   $\overrightarrow{g}\left(k\right);$	▹Best Found Solution

Next, the particles search begins. For each dimension of each particle, an element $v_{i,d}$ is determined as follows:

$$v_{i,d}(k+1)=w{v}_{i,d}\left(k\right)+{\phi }_p{r}_p\left(p_{i,d}\left(k\right)-x_{i,d}\left(k\right)\right)+{\phi }_g{r}_g\left(g_d\left(k\right)-x_{i,d}\left(k\right)\right),$$

(1)

where the parameters $r_p$ and $r_g$ are uniformly distributed random numbers in the range $[0,1]$, i.e., $U(0,1)$. In the first part of Equation (1), the inertial weight w represents the amount of momentum the particles have. The second part is the “cognition” part, which represents the independent behavior of each particle. The final part is the “social” part, which represents the collaboration among the particles. The constants ${\phi }_p$ and ${\phi }_g$ determine the relative influences of the cognition and social parts, and eventually, pull each particle toward position $p_{best}$ and $g_{best}$.

The next position of particle i, ${\overrightarrow{x}}_i(k+1)$, is calculated using Equation (1) as follows:

$${\overrightarrow{x}}_i(k+1)={\overrightarrow{x}}_i\left(k\right)+{\overrightarrow{v}}_i(k+1).$$

(2)

Subsequently, if the value of $f\left({\overrightarrow{x}}_i\left(k\right)\right)$ is smaller than the value of $f\left({\overrightarrow{p}}_i\left(k\right)\right)$, then the vector ${\overrightarrow{p}}_i\left(k\right)$ is updated to ${\overrightarrow{x}}_i\left(k\right)$. In addition, if the value of $f\left({\overrightarrow{p}}_i\left(k\right)\right)$ is smaller than the value of $f\left(\overrightarrow{g}\right(k\left)\right)$, then, the vector $\overrightarrow{g}\left(k\right)$ is updated to ${\overrightarrow{p}}_i\left(k\right)$.

This process is repeated until a termination criterion is met, e.g., the maximum number of iterations or a solution with an acceptable objective function value is reached.

2.2. The Bare-Bones PSO (BBPSO) Algorithm

As mentioned above, the BBPSO algorithm does not have to set up the parameters w, ${\phi }_p$, and ${\phi }_g$, unlike the standard PSO algorithm. In the BBPSO algorithm, the next position of a particle is determined by sampling a Gaussian distribution whose mean is given by the average globally best position of the swarm(s), $g_{best}$, and the personally-best position of the particle, $p_{best}$ and whose the standard deviation given by the absolute difference between $g_{best}$ and $p_{best}$. For each element (or dimension) of a particle, the next position is determined in the BBPSO algorithm using the following equations instead of using Equations (1) and (2), which are for the standard PSO algorithm:

$$x_{i,d}(k+1)=N\left({\mu }_{i,d}\left(k\right),{\sigma }_{i,d}^2\left(k\right)\right)$$

(3)

$${\mu }_{i,d}\left(k\right)={\displaystyle \frac{g_d\left(k\right)+p_{i,d}\left(k\right)}{2}}$$

(4)

$${\sigma }_{i,d}\left(k\right)=\left|g_d\left(k\right)-p_{i,d}\left(k\right)\right|,$$

(5)

where $N\left({\mu }_{i,d}\left(k\right),{\sigma }_{i,d}^2\left(k\right)\right)$ is a random number generator based on a Gaussian distribution with mean ${\mu }_{i,d}\left(k\right)$ in Equation (4) and standard deviation ${\sigma }_{i,d}\left(k\right)$ in Equation (5) for the d-th dimension of the particle i. Except for the above step, the procedure is equivalent to that of the standard PSO algorithm.

2.3. The Cooperative Approach

Generally, in population-based algorithms, including the PSO algorithm, an agent in one population represents an intact n-dimensional candidate solution. In the standard PSO algorithm, there is one swarm of $N_P$ particles, each of which has n components, that attempts to find an optimal n-dimensional solution. However, in this case, the algorithm frequently undergoes the “two steps forward, one step backward” phenomenon described in [20], especially when it is solving a high-dimensional problem. This appearance of this phenomenon means that although the fitness of a particle (or a candidate solution vector) may be considerably improved during the next time step, some of its components may have been changed from a better value to a rather poor value. Indeed, an improvement in two components (two steps forward) overrules a potentially good value for a single component (one step backward) for a problem with a three-dimensional solution vector. Eventually, valuable information is unintentionally lost.

One solution to the “two steps forward, one step backward” problem is to evaluate the objective function more frequently, perhaps each time a component in the candidate solution vector is updated, which results in much quicker feedback. However, in this case a problem remains. The function is only evaluated for a complete n-dimensional solution vector. Therefore, after a specific component is updated, the values of the $n-1$ other components of the candidate vector still need to be chosen. One method of overcoming these problems proposed in [20], the Cooperative PSO (CPSO) algorithm, employs a cooperative approach. In the CPSO algorithm, unlike the standard PSO algorithm, the solution vector is split into its components so that K swarms of $N_P$ particles containing $\lceil n/K\rceil$-dimensional or $\lfloor n/K\rfloor$-dimensional components, where K is a pre-determined parameter called the split factor, are optimized. This approach effectively increases the solution diversity and the amount of information exchanged to avoid the “two steps forward, one step backward” phenomenon.

2.4. The Jumping Strategy

When variants of the PSO algorithm are applied to optimization problems with many local optima in a high-dimensional search space, they may become stuck at the local optima. The jumping strategy [17,21,22,23] was proposed to escape from local optima, and promising results have been obtained. This strategy has been implemented as a mutation operator in Evolutionary Algorithms (EAs) based on a Gaussian and Cauchy probability distributions.

The goal of the jumping strategy is to allow particles in PSO algorithms to escape from local optima to which they are prematurely attracted. The motion of the particles in this situation stagnates with no improvement in their fitness. Whether a particle is stagnating can be determined by monitoring its fitness, and then, the stagnating particles move to a new point that is selected using the jumping strategy. This aim can be accomplished by introducing a stagnation interval ($C_{F,j}$ for each particle in this paper), which monitors the fitness of each particle, and increasing it by one during each iteration until its value reaches a pre-determined maximum number of iterations, which is called the maximum stagnation interval in [17] ($M_F$ in this paper).

When the particle should jump to a new point, its next position is determined by choosing between Gaussian and Cauchy jumps as follows:

$${\overrightarrow{x}}_i(k+1)={\overrightarrow{p}}_i\left(k\right)·\left(1+\eta ·N(0,1)\right)$$

(6)

$${\overrightarrow{x}}_i(k+1)={\overrightarrow{p}}_i\left(k\right)·\left(1+\eta ·C(0,1)\right)$$

(7)

where the parameter $\eta$ is for scaling and the vector ${\overrightarrow{p}}_i\left(k\right)$ is the best known position of particle i. In Equation (6), $N(0,1)$ is a random number generated using a Gaussian probability distribution with a mean of 0 and a standard deviation of 1. In Equation (7), $C(0,1)$ is a random number generated using a Cauchy probability distribution with $\gamma =1$ centered at the origin and described by

$$f\left(x\right)=\frac{\gamma }{\pi \left({\gamma }^2+x^2\right)},-\infty <x<\infty .$$

(8)

3. Heterogeneous Cooperative BBPSO with Jumping (HCBBPSO-Jx) Algorithms

Algorithm 2 shows detailed pseudo-code for the Heterogeneous Cooperative BBPSO with the jumping (HCBBPSO-Jx) algorithms proposed in this paper. The proposed algorithms consist of three main parts: a cooperative BBPSO step, a BBPSO with jumping step, and a cooperative part in which information is exchanged between two heterogeneous algorithms. First, the parameters for the HCBBPSO-Jx algorithms are initialized. The matrices $\mathbf{x}$ and $\mathbf{y}$ contain the current position vectors of the particles in swarms P and swarm Q, respectively. Additionally, the matrices $\mathbf{p}$ and $\mathbf{q}$ (the bottom matrices in Figure 2) include the $p_{best}$ information of swarms P and swarm Q, respectively. The vector ${\overrightarrow{g}}_P$ (the top vectors in Figure 2) stores the $g_{best}$ information of swarms P and the vector ${\overrightarrow{g}}_Q$ stores that of swarm Q.

3.1. The Cooperative BBPSO (CBBPSO) Step

Once the necessary parameters for the HCBBPSO-Jx algorithms have been initialized, the Cooperative BBPSO (CBBPSO) step is performed. This step introduces the aforementioned cooperative approach. It has been modified for use in high-dimensional problems as shown in lines 7–18 of Algorithm 2. Unlike the CPSO algorithm [20], the HCBBPSO-Jx algorithms are based on the BBPSO algorithm instead of the traditional PSO algorithm, i.e., they use the equation of motion from the BBPSO algorithm, Equation (3), to move the particles. This method is simpler and more robust than that of the PSO algorithm. Additionally, the algorithm uses the P swarms, which are K swarms of $N_P$ particles, as shown in Figure 2a. Here, the constant K is a pre-determined parameter called the split factor, as it is in the CPSO algorithm. From the constant K, the parameter $K_1$ is calculated as $K_1=mod(n,K)$. Then, the parameter $K_2$ is $K-K_1$. Of the KP swarms, $K_1$ contain particles that have $K_C$ dimensions, where $K_C=\lceil n/K\rceil$. The particles in the remaining $K_2$ swarms have $K_F$-dimensional components, where $K_F=\lfloor n/K\rfloor$. That is to say, an n-dimensional solution vector is divided into $K_1$ $K_C$-dimensional components and $K_2$ $K_F$-dimensional components, where $n=K_C\times K_1+K_F\times K_2$. In addition, to reduce the Number of Function Evaluations (NFEs), the proposed algorithm uses a double if statement for updates (lines 10–15 of Algorithm 2).

As in [20], for cooperation (or, more precisely, information exchange) between swarms, a “blackboard,” which is a shared memory in which particles can post or read hints, is used. To establish this blackboard, the algorithm introduces a context vector, shown at the top of Figure 2a, which selects the globally best particle from each of the K $\mathbf{P}$ swarms, and is used to evaluate the particles. To evaluate all of the particles in the s-th swarm, the other $n-K_X$ components in the context vector are kept constant while the $1+(s-1)K_X$-th to the $s{K}_X$-th components of the context vector are replaced by each particle from the s-th swarm in turn. The function $\overrightarrow{B}(s,z_s)$ shown (and in lines 10 and 12 of Algorithm 2) plays this role to create an n-dimensional vector that is evaluated.

$$\overrightarrow{B}(s,z_s)\equiv \left(g_{P,1},\cdots ,g_{P,(s-1)K_X},z_s,g_{P,s{K}_X},\cdots ,g_{P,n}\right)$$

(9)

The subscript X of $K_X$ is determined as follows: for the s-th swarm, if $s\in \{1,\cdots ,K_1\}$ then $X=C$ and $X=F$ if $s\in \{K_1+1,\cdots ,K\}$.

3.2. The BBPSO with Jumping (BBPSO-Jx) Step

As the second step, to achieve robust heterogeneous cooperation for solving high-dimensional problems, the proposed HCBBPSO-Jx algorithms introduce the BBPSO algorithm with a jumping strategy. Figure 2b displays the configuration of the $g_{best}$ and $p_{best}$ of the swarm Q, which is the general form of the PSO variants. The standard BBPSO algorithm may become stuck at a local optimum when solving a high-dimensional problem with many local optima. In this case, as mentioned in Section 2.4, the jumping strategy enables particles to escape from the local optima. If there is no improvement for the j-th particle, then, the update failure counter, $C_{F,j}$, is increased by one during each iteration until it reaches the predefined maximum allowable number of update failures, $M_F$. After reaching the maximum number of failures, $M_F$, the particle jumps to a new point using Equation (6) or (7). After the performances of two cases, which use Gaussian and Cauchy probability distributions, have been compared, the best jumping strategy for the proposed HCBBPSO-Jx algorithms is identified in Section 4.

Algorithm 2 HCBBPSO-Jx Algorithms (for x, C: Cauchy, G: Gaussian).
1:$SetParameters(N_P,K,N_Q,\eta ,M_F);$	▹Set parameters
▹K: split factor, $M_F$: # of maximum allowable update failure, $\eta$: jump scaling factor, n: dimension of given problem
2:$k=1;$
3:$K_1=mod(n,K);$$K_2=K-K_1;$$K_C=ceil(n/K);$$K_F=floor(n/K);$
▹For spliting into solution vector components for CBBPSO
4:$Initialize(\mathbf{x}\left(1\right),\mathbf{p}\left(1\right),{\overrightarrow{g}}_P\left(1\right),\mathbf{y}\left(1\right),\mathbf{q}\left(1\right),{\overrightarrow{g}}_Q\left(1\right),{\overrightarrow{C}}_F);$
▹Initialize swarms P and swarm Q, ${\overrightarrow{C}}_F$: Counter vector for update failure
5: while   Termination condition does not meet do
6:      $k=k+1;$
$▾▾$Cooperative BBPSO Step$▾▾$
7:      for   $s=1$ to   K
▹For s-th swarm with $K_C$ dimensions if $s\in \{1,\cdots ,K_1\}$ and $K_F$ dimensions if $s\in \{K_1+1,\cdots ,K\}$
8:           $D=[1+(s-1)K_X:s{K}_X];$	▹$X=C$ if $s\in \{1,\cdots ,K_1\}$ and $X=F$ if $s\in \{K_1+1,\cdots ,K\}$
9:           for   $i=1$ to $N_P$ do	▹For each particle
10:                if   $f(\overrightarrow{B}(D,x_{i,D}\left(k\right)))<f(\overrightarrow{B}(D,p_{i,D}\left(k\right)))$ then
11:                     $p_{i,D}\left(k\right)\leftarrow x_{i,D}\left(k\right);$	▹Update $p_{best}$ components (or particle) of swarms P
12:                     if   $f(\overrightarrow{B}(D,p_{i,D}\left(k\right)))<f(\overrightarrow{B}(D,g_{P,D}\left(k\right)))$ then
13:                          $g_{P,D}\left(k\right)\leftarrow p_{i,D}\left(k\right);$	▹Update $g_{best}$ components of swarms P ($K_C$ or $K_F$ components of context vector)
14:                     end if
15:                end if
16:           end for
17:           ${\overrightarrow{x}}_D\left(k\right)\sim N({\overrightarrow{\mu }}_D\left(k\right),{\overrightarrow{\sigma }}_D^2\left(k\right));$	▹Moving swarm using Equations (3)–(5) for swarms P
18:      end for
$▾▾$BBPSO with Jump Step$▾▾$
19:      for   $j=1$ to   $N_Q$ do	▹For each particle
20:           if   $C_{F,j}\le M_F$ then	▹Perform original BBPSO
21:                ${\overrightarrow{y}}_j\left(k\right)\sim N({\overrightarrow{\mu }}_j\left(k\right),{\overrightarrow{\sigma }}_j^2\left(k\right));$	▹Moving swarm using Equations (3)–(5) for swarm Q
22:           else	▹Perform jump
23:                $C_{F,j}\leftarrow 0;$
24:                ${\overrightarrow{y}}_j\left(k\right)\sim {\overrightarrow{q}}_j\left(k\right)·\left(1+\eta ·C(0,1)\right);$	▹$C(0,1)$ for Cauchy jump or $N(0,1)$ for Gaussian jump
25:           end if
26:           for   $s=1$ to   K do	▹Information exchange from swarms P to swarm Q
27:                $D=[1+(s-1)K_X:s{K}_X];$	▹$X=C$ if $s\in \{1,\cdots ,K_1\}$ and $X=F$ if $s\in \{K_1+1,\cdots ,K\}$
28:                $m\sim U(1,N_Q);$	▹Integer uniform distribution with range $[1,N_P]$
29:                $y_{m,D}\left(k\right)\leftarrow g_{P,D}\left(k\right);$
30:           end for
31:           if   $f\left({\overrightarrow{y}}_j\left(k\right)\right)<f\left({\overrightarrow{q}}_j\left(k\right)\right)$ then
32:                ${\overrightarrow{q}}_j\left(k\right)\leftarrow {\overrightarrow{y}}_j\left(k\right);$	▹Update $p_{best}$ vector (or particle) of swarm Q
33:                if   $f\left({\overrightarrow{q}}_j\left(k\right)\right)<f\left({\overrightarrow{g}}_Q\left(k\right)\right)$ then
34:                     ${\overrightarrow{g}}_Q\left(k\right)\leftarrow {\overrightarrow{q}}_j\left(k\right);$	▹Update $g_{best}$ vector of swarm Q
35:                end if
36:           else
37:                $C_{F,j}\leftarrow C_{F,j}+1;$
38:           end if
39:      end for
40:      for   $d=1$ to   n do	▹Information exchange from swarm Q to swarms P
41:           $m\sim U(1,N_P);$	▹Integer uniform distribution with range $[1,N_Q]$
42:           $x_{m,d}\left(k\right)\leftarrow g_{Q,d}\left(k\right);$
43:      end for
44:      $\overrightarrow{g}\left(k\right)\leftarrow min({\overrightarrow{g}}_P\left(k\right),{\overrightarrow{g}}_Q\left(k\right))$
45:end while
46: return   $\overrightarrow{g}\left(k\right);$	▹Best found solution

3.3. The Steps Involving Cooperation by Exchanging Information between the CBBPSO and BBPSO-Jx Algorithms

The final part of the HCBBPSO-Jx algorithms is where heterogeneous cooperation between the CBBPSO step and the BBPSO with jumping step occurs in the HCBBPSO-Jx algorithms; information from the previous two steps is exchanged, as shown in lines 26–30 and 40–43 of Algorithm 2. In this step, information is exchanged once per iteration when n components that are randomly selected from the matrices $\mathbf{x}$ and $\mathbf{y}$, each corresponding to a component of its $g_{best}$, are substituted. This step helps increase the diversity of the solutions searched.

Like other variants of the PSO algorithm, the above three-step process is repeated until a termination criterion is met, e.g., the maximum number of iterations or a solution with an acceptable objective function value is reached.

4. Comparative Simulations

This section presents the results of the simulations and a discussion of comparable simulations performed using five variants of the PSO algorithm. The goal is to verify the performance of the proposed HCBBPSO-Jx algorithms by applying them to the 20 1000-dimensional benchmark functions from the CEC 2010 Special Session. The CEC 2010 Special Session on Large-Scale Global Optimization (LSGO) identified 20 benchmark problems [24] that provide convenience and flexibility for comparing various optimization algorithms that are specifically designed for large-scale global optimization. The test suite includes four types of high-dimensional problem: (1) separable functions; (2) partially separable functions, which have a small number of dependent variables, and all of the remaining variables are independent; (3) partially separable functions that consist of multiple independent subcomponents, each of which is m-nonseparable; and (4) fully nonseparable functions. The detailed mathematical formulas for and properties of these functions are described in [24]. Section 4.1 describes the simulation environment and setup. Section 4.2 describes the comparative results evaluated using the Formula One point system, which is the method used in the LSGO challenge posed in the CEC 2010 competition. Finally, Section 4.3 is dedicated to reporting the results of the best algorithm tested in the comparative simulations using the method that represented the results of the LSGO competition.

4.1. The Simulation Environment and Setup

The simulations were conducted using the 20 1000-dimensional minimization problems from CEC 2010. For each problem, the simulation was run 25 times for statistical accuracy. The simulation terminated when the maximum number of function evaluations (MaxNFEs), which was set to $3\times {10}^6$ for all of the trials, was reached. The simulator and algorithm used in this simulation were implemented in MATLAB for 32-bit Windows 8.1. All of the simulations were performed on four computers with an Intel ⓐ 3.07 GHz processor and 4 GB of RAM. For fairness, each computer only ran simulations for the following pre-assigned functions: $f_1$∼$f_5$ on COM1, $f_6$∼$f_{10}$ on COM2, $f_{11}$∼$f_{15}$ on COM3, and $f_{16}$∼$f_{20}$ on COM4.

For comparison, the five variants of the PSO algorithm included the two groups of algorithms shown in

Table 1. Parameter setting of each algorithm for comparative simulation with Particle Swarm Optimization (PSO) variants.

Algorithm	Parameter	Value
HCBBPSO-Jx	Number of particles in Swarms P ($N_P$)	25
(x = C or G)	Split factor (K)	50
	Number of particles in Swarm Q ($N_Q$)	25
	Jump scaling factor ($\eta$)	1.1
	Number of maximum allowable update failure ($M_F$)	5
CPSO-HK [20]	Number of particles in each algorithm (CPSO-SK and PSO respectively)	25
	Split factor (K)	6
	Weight for previous velocity (w)	0.72
	Coefficient of cognition term ($c_1$)	1.49
	Coefficient of social term ($c_2$)	1.49
BBPSOjumpx [17]	Number of particles	50
(x = C or G)	Jump scaling factor ($\eta$)	1.1
	Number of maximum allowable update failure ($M_F$)	5
APSO [25]	Number of particles	50
	Weight for previous velocity	0.9
	Coefficient of cognition term	2
	Coefficient of social term	2
CLPSO [26]	Number of particles	50
	$w_0$	0.9
	$w_1$	0.4
	Coefficient of cognition term	1.49445
	Coefficient of social term	1.49445
	Refreshing gap (m)	7
	$V_{max}=-V_{min}$ ($x_L^i$, $x_U^i$: Lower and upper bound of i-th dimension)	0.2($x_U^i-x_L^i$)

. The first group included three algorithms that are associated with the proposed HCBBPSO-Jx algorithms: The cooperative PSO-H_K (CPSO-H_K) algorithm, which was the most robust of the CPSO variants proposed in [20] and was obtained by combining the CPSO-S_K algorithm with the PSO algorithm and Bare-Bones PSO with Cauchy and Gaussian jumping (BBPSOjumpC and BBPSOjumpG, respectively) algorithms [17], which improved the ability of the BBPSO algorithm to escape from local optima. The second comprised well-known variants of the PSO algorithm. The Adaptive PSO (APSO) algorithm [25] used a parameter adaptation scheme and elitist learning strategy to improve the PSO algorithm. The Comprehensive Learning PSO (CLPSO) algorithm [26] improved the diversification ability of the PSO algorithm by using comprehensive learning, in which all of the historically best information on the other particles was used to update a particle’s velocity. All of the algorithms used the same parameters in all of the simulations; these are shown in the table. The parameters for each algorithm were assigned the values that resulted in the best performance and that were recommended in the literature. In addition, to achieve a fair test, the initial population size was set to 50 for each algorithm. All of the search parameters were initialized using a uniform random process within the search space.

4.2. The Results of Comparing the Simulations Performed by the PSO Algorithm Variants

Table 2. The results of applying the scoring system from the CEC 2010 LSGO Challenge to the results of the simulations performed by variants of the PSO algorithm.

Ranking	1st	2nd	3rd	4th	5th	6th	7th
Algorithm	HCBBPSO-JG	HCBBPSO-JC	BBPSOjumpC	BBPSOjumpG	CLPSO	CPSO6	APSO
Total Score	5474	5357	4276	4233	4131	2517	2212

shows the results of the simulations performed by the variants of the PSO algorithm after evaluation using the scoring system from the CEC 2010 LSGO Challenge. The scoring system used in the CEC 2010 LSGO Challenge was as follows: for each algorithm, a table of the type shown in Table 7 that contains 300 competition categories was formed. The competition categories were 20 functions ($f_1$∼$f_{20}$), 3 limits on the NFEs ($1.2\times {10}^5,6.0\times {10}^5$, and $3.0\times {10}^6$), and 5 statistical values (best, median, worst, mean, and standard deviation) at each limit on the NFEs for each of the 25 runs. Then, the LSGO Challenge applies the Formula One point system to the data from the challenge participants in each of the 300 categories.

Table 3. The results of applying the scoring system from the CEC 2010 LSGO Challenge to the results of the simulations performed by variants of the PSO algorithm.

Ranking	1st	2nd	3rd	4th	5th	6th	7th	8th	9th	10th
Points	25	18	15	12	10	8	6	4	2	1

shows the points for each ranking in the Formula One point system. The winner received 25 points and other rankers received differentiated points according to ranking. Like the CEC 2010 LSGO Challenge, in this simulation, the smaller measured values in all of the categories, the higher the ranking and the more points. In particular, a small standard deviation means that the performance was more reliable. Eventually, the participant with the highest total score wins. In the results of the evaluation using this scoring system, the proposed HCBBPSO-JG algorithm was the best of the participating algorithms. In addition, the HCBBPSO-JG algorithm performed better than the HCBBPSO-JC algorithm that was proposed alongside it.

Table 4. A comparison of the results of simulations performed by variants of the PSO algorithm. The best, median, worst, mean, standard deviation, p-value, and significance sign of the 25 runs when the NFEs counter reached $3.0\times {10}^6$ for the functions $f_1$∼$f_7$ are reported. The p-value was determined using the Wilcoxon signed-rank test between the best algorithm (HCBBPSO-JG) and the others. The significance sign “+” means that the HCBBPSO-JG algorithm was significantly better than the algorithm compared to it, the sign “∼” means that the two were not significantly different, and the sign “−” means that the HCBBPSO-JG algorithm was significantly worse than the algorithm compared to it. The measured value written in bold in colored cells for each function represents the value of the best algorithm for each statistical value.

Algorithm		${\mathit{f}}_1$	${\mathit{f}}_2$	${\mathit{f}}_3$	${\mathit{f}}_4$	${\mathit{f}}_5$	${\mathit{f}}_6$	${\mathit{f}}_7$
APSO	Best	$5.92\times {10}^5$	$2.37\times {10}^4$	$1.99\times {10}^1$	$8.91\times {10}^{12}$	$2.94\times {10}^8$	$2.26\times {10}^6$	$1.86\times {10}^8$
	Median	$7.66\times {10}^6$	$2.42\times {10}^4$	$2.15\times {10}^1$	$1.55\times {10}^{13}$	$3.85\times {10}^8$	$1.98\times {10}^7$	$5.23\times {10}^8$
	Worst	$3.63\times {10}^{11}$	$2.44\times {10}^4$	$2.15\times {10}^1$	$2.96\times {10}^{13}$	$6.55\times {10}^8$	$2.11\times {10}^7$	$1.38\times {10}^9$
	Mean	$7.02\times {10}^{10}$	$2.42\times {10}^4$	$2.13\times {10}^1$	$1.67\times {10}^{13}$	$4.02\times {10}^8$	$1.49\times {10}^7$	$6.45\times {10}^8$
	Std	$1.45\times {10}^{11}$	$2.17\times {10}^2$	$4.30\times {10}^{-1}$	$5.40\times {10}^{12}$	$9.82\times {10}^7$	$8.31\times {10}^6$	$3.68\times {10}^8$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$8.54\times {10}^{-4}$	$6.39\times {10}^{-1}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(+)	(+)	(−)	(∼)	(+)
BBPSOjumpC	Best	$1.24\times {10}^6$	$4.21\times {10}^3$	$1.93\times {10}^1$	$1.49\times {10}^{12}$	$9.45\times {10}^7$	$1.03\times {10}^6$	$7.77\times {10}^4$
	Median	$9.58\times {10}^6$	$4.50\times {10}^3$	$1.95\times {10}^1$	$2.71\times {10}^{12}$	$1.44\times {10}^8$	$2.20\times {10}^6$	$1.67\times {10}^5$
	Worst	$4.92\times {10}^7$	$4.91\times {10}^3$	$1.97\times {10}^1$	$5.61\times {10}^{12}$	$\mathbf{2}.\mathbf{02}\times {\mathbf{10}}^{\mathbf{8}}$	$3.44\times {10}^6$	$7.33\times {10}^5$
	Mean	$1.42\times {10}^7$	$4.54\times {10}^3$	$1.95\times {10}^1$	$3.21\times {10}^{12}$	$1.45\times {10}^8$	$2.19\times {10}^6$	$2.24\times {10}^5$
	Std	$1.51\times {10}^7$	$2.26\times {10}^2$	$1.08\times {10}^{-1}$	$1.23\times {10}^{12}$	$3.59\times {10}^7$	$7.20\times {10}^5$	$1.71\times {10}^5$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$3.03\times {10}^{-1}$
	(Sign)	(+)	(+)	(+)	(+)	(−)	(−)	(∼)
BBPSOjumpG	Best	$6.16\times {10}^5$	$4.34\times {10}^3$	$1.91\times {10}^1$	$1.55\times {10}^{12}$	$\mathbf{8}.\mathbf{96}\times {\mathbf{10}}^{\mathbf{7}}$	$\mathbf{1}.\mathbf{99}\times {\mathbf{10}}^{\mathbf{1}}$	$5.86\times {10}^4$
	Median	$4.85\times {10}^6$	$4.62\times {10}^3$	$1.93\times {10}^1$	$3.17\times {10}^{12}$	$\mathbf{1}.\mathbf{30}\times {\mathbf{10}}^{\mathbf{8}}$	$2.26\times {10}^6$	$1.87\times {10}^5$
	Worst	$5.05\times {10}^7$	$4.89\times {10}^3$	$1.95\times {10}^1$	$7.85\times {10}^{12}$	$2.13\times {10}^8$	$4.11\times {10}^6$	$5.92\times {10}^5$
	Mean	$1.11\times {10}^7$	$4.63\times {10}^3$	$1.94\times {10}^1$	$3.36\times {10}^{12}$	$\mathbf{1}.\mathbf{41}\times {\mathbf{10}}^{\mathbf{8}}$	$2.38\times {10}^6$	$2.28\times {10}^5$
	Std	$1.31\times {10}^7$	$1.48\times {10}^2$	$1.28\times {10}^{-1}$	$1.56\times {10}^{12}$	$4.24\times {10}^7$	$9.76\times {10}^5$	$1.60\times {10}^5$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$7.30\times {10}^{-2}$
	(Sign)	(+)	(+)	(+)	(+)	(−)	(−)	(∼)
CLPSO	Best	$3.40\times {10}^8$	$\mathbf{2}.\mathbf{36}\times {\mathbf{10}}^{\mathbf{3}}$	$1.44\times {10}^1$	$9.67\times {10}^{12}$	$1.45\times {10}^8$	$1.64\times {10}^2$	$4.03\times {10}^8$
	Median	$4.20\times {10}^8$	$\mathbf{2}.\mathbf{50}\times {\mathbf{10}}^{\mathbf{3}}$	$1.47\times {10}^1$	$1.44\times {10}^{13}$	$1.89\times {10}^8$	$\mathbf{1}.\mathbf{95}\times {\mathbf{10}}^{\mathbf{6}}$	$6.80\times {10}^8$
	Worst	$5.57\times {10}^8$	$\mathbf{2}.\mathbf{76}\times {\mathbf{10}}^{\mathbf{3}}$	$1.49\times {10}^1$	$1.80\times {10}^{13}$	$2.46\times {10}^8$	$\mathbf{3}.\mathbf{24}\times {\mathbf{10}}^{\mathbf{6}}$	$1.62\times {10}^9$
	Mean	$4.24\times {10}^8$	$\mathbf{2}.\mathbf{50}\times {\mathbf{10}}^{\mathbf{3}}$	$1.47\times {10}^1$	$1.38\times {10}^{13}$	$1.99\times {10}^8$	$\mathbf{1}.\mathbf{75}\times {\mathbf{10}}^{\mathbf{6}}$	$7.10\times {10}^8$
	Std	$5.95\times {10}^7$	$\mathbf{8}.\mathbf{99}\times {\mathbf{10}}^{\mathbf{1}}$	$1.41\times {10}^{-1}$	$2.13\times {10}^{12}$	$\mathbf{3}.\mathbf{06}\times {\mathbf{10}}^{\mathbf{7}}$	$1.04\times {10}^6$	$2.93\times {10}^8$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-4}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(−)	(+)	(+)	(−)	(−)	(+)
CPSO-H6	Best	$1.16\times {10}^{10}$	$1.27\times {10}^4$	$2.04\times {10}^1$	$1.18\times {10}^{13}$	$3.98\times {10}^8$	$1.88\times {10}^7$	$2.09\times {10}^9$
	Median	$1.80\times {10}^{10}$	$1.37\times {10}^4$	$2.05\times {10}^1$	$4.14\times {10}^{13}$	$4.99\times {10}^8$	$1.94\times {10}^7$	$9.69\times {10}^9$
	Worst	$2.84\times {10}^{10}$	$1.55\times {10}^4$	$2.05\times {10}^1$	$1.08\times {10}^{14}$	$6.15\times {10}^8$	$1.98\times {10}^7$	$1.78\times {10}^{10}$
	Mean	$1.83\times {10}^{10}$	$1.39\times {10}^4$	$2.05\times {10}^1$	$4.19\times {10}^{13}$	$4.94\times {10}^8$	$1.94\times {10}^7$	$8.70\times {10}^9$
	Std	$5.47\times {10}^9$	$8.98\times {10}^2$	$\mathbf{5}.\mathbf{62}\times {\mathbf{10}}^{-\mathbf{2}}$	$2.50\times {10}^{13}$	$6.19\times {10}^7$	$2.96\times {10}^5$	$4.87\times {10}^9$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$7.30\times {10}^{-2}$	$2.01\times {10}^{-3}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(+)	(+)	(∼)	(−)	(+)
HCBBPSO-JC	Best	$7.01\times {10}^{-20}$	$2.39\times {10}^3$	$2.66\times {10}^0$	$3.29\times {10}^{11}$	$4.00\times {10}^8$	$1.88\times {10}^7$	$\mathbf{1}.\mathbf{39}\times {\mathbf{10}}^{\mathbf{4}}$
	Median	$3.63\times {10}^{-18}$	$2.72\times {10}^3$	$\mathbf{3}.\mathbf{21}\times {\mathbf{10}}^{\mathbf{0}}$	$\mathbf{7}.\mathbf{61}\times {\mathbf{10}}^{\mathbf{11}}$	$5.57\times {10}^8$	$1.98\times {10}^7$	$2.37\times {10}^5$
	Worst	$\mathbf{1}.\mathbf{01}\times {\mathbf{10}}^{-\mathbf{16}}$	$2.92\times {10}^3$	$\mathbf{4}.\mathbf{41}\times {\mathbf{10}}^{\mathbf{0}}$	$\mathbf{8}.\mathbf{30}\times {\mathbf{10}}^{\mathbf{11}}$	$7.33\times {10}^8$	$1.99\times {10}^7$	$2.69\times {10}^5$
	Mean	$\mathbf{1}.\mathbf{57}\times {\mathbf{10}}^{-\mathbf{17}}$	$2.65\times {10}^3$	$\mathbf{3}.\mathbf{20}\times {\mathbf{10}}^{\mathbf{0}}$	$\mathbf{6}.\mathbf{61}\times {\mathbf{10}}^{\mathbf{11}}$	$5.66\times {10}^8$	$1.97\times {10}^7$	$2.08\times {10}^5$
	Std	$\mathbf{3}.\mathbf{47}\times {\mathbf{10}}^{-\mathbf{17}}$	$1.68\times {10}^2$	$5.67\times {10}^{-1}$	$\mathbf{1}.\mathbf{83}\times {\mathbf{10}}^{\mathbf{11}}$	$1.01\times {10}^8$	$2.61\times {10}^5$	$7.51\times {10}^4$
	p-value	$9.34\times {10}^{-1}$	$1.07\times {10}^{-1}$	$4.54\times {10}^{-1}$	$3.30\times {10}^{-1}$	$5.24\times {10}^{-1}$	$1.69\times {10}^{-1}$	$2.15\times {10}^{-2}$
	(Sign)	(∼)	(∼)	(∼)	(∼)	(∼)	(∼)	(+)
HCBBPSO-JG	Best	$\mathbf{4}.\mathbf{16}\times {\mathbf{10}}^{-\mathbf{20}}$	$2.51\times {10}^3$	$\mathbf{2}.\mathbf{08}\times {\mathbf{10}}^{\mathbf{0}}$	$\mathbf{2}.\mathbf{93}\times {\mathbf{10}}^{\mathbf{11}}$	$4.21\times {10}^8$	$1.95\times {10}^7$	$3.71\times {10}^4$
	Median	$\mathbf{7}.\mathbf{76}\times {\mathbf{10}}^{-\mathbf{19}}$	$2.80\times {10}^3$	$3.40\times {10}^0$	$8.84\times {10}^{11}$	$5.58\times {10}^8$	$1.98\times {10}^7$	$\mathbf{1}.\mathbf{66}\times {\mathbf{10}}^{\mathbf{5}}$
	Worst	$2.63\times {10}^{-16}$	$3.04\times {10}^3$	$5.18\times {10}^0$	$1.38\times {10}^{12}$	$7.30\times {10}^8$	$1.99\times {10}^7$	$\mathbf{2}.\mathbf{45}\times {\mathbf{10}}^{\mathbf{5}}$
	Mean	$2.81\times {10}^{-17}$	$2.77\times {10}^3$	$3.47\times {10}^0$	$7.82\times {10}^{11}$	$5.54\times {10}^8$	$1.97\times {10}^7$	$\mathbf{1}.\mathbf{45}\times {\mathbf{10}}^{\mathbf{5}}$
	Std	$6.88\times {10}^{-17}$	$1.63\times {10}^2$	$8.55\times {10}^{-1}$	$3.23\times {10}^{11}$	$9.93\times {10}^7$	$\mathbf{1}.\mathbf{08}\times {\mathbf{10}}^{\mathbf{5}}$	$\mathbf{7}.\mathbf{32}\times {\mathbf{10}}^{\mathbf{4}}$
Total # of +/∼/−		5/1/0	4/1/1	5/1/0	5/1/0	0/2/4	0/2/4	4/2/0

,

Table 5. A comparison of the results of simulations performed by variants of the PSO algorithm. The best, median, worst, mean, standard deviation, p-value, and significance sign of the 25 runs when the NFEs counter reached $3.0\times {10}^6$ for the functions $f_8$∼$f_{14}$ are reported. The p-value was determined using the Wilcoxon signed-rank test between the best algorithm (HCBBPSO-JG) and the others. The significance sign “+” means that the HCBBPSO-JG algorithm was significantly better than the algorithm compared to it, the sign “∼” means that the two were not significantly different, and the sign “−” means that the HCBBPSO-JG algorithm was significantly worse than the algorithm compared to it. The measured value written in bold in colored cells for each function represents the value of the best algorithm for each statistical value.

Algorithm		${\mathit{f}}_8$	${\mathit{f}}_9$	${\mathit{f}}_{10}$	${\mathit{f}}_{11}$	${\mathit{f}}_{12}$	${\mathit{f}}_{13}$	${\mathit{f}}_{14}$
APSO	Best	$2.51\times {10}^8$	$3.20\times {10}^{11}$	$2.39\times {10}^4$	$2.18\times {10}^2$	$4.33\times {10}^5$	$3.06\times {10}^{12}$	$3.11\times {10}^{11}$
	Median	$1.07\times {10}^9$	$3.76\times {10}^{11}$	$2.43\times {10}^4$	$2.27\times {10}^2$	$5.67\times {10}^5$	$3.43\times {10}^{12}$	$4.24\times {10}^{11}$
	Worst	$2.59\times {10}^{10}$	$4.19\times {10}^{11}$	$2.48\times {10}^4$	$2.36\times {10}^2$	$3.84\times {10}^6$	$3.73\times {10}^{12}$	$4.49\times {10}^{11}$
	Mean	$3.70\times {10}^9$	$3.73\times {10}^{11}$	$2.43\times {10}^4$	$2.27\times {10}^2$	$8.47\times {10}^5$	$3.43\times {10}^{12}$	$4.13\times {10}^{11}$
	Std	$6.47\times {10}^9$	$2.56\times {10}^{10}$	$3.21\times {10}^2$	$6.02\times {10}^0$	$8.83\times {10}^5$	$1.56\times {10}^{11}$	$3.67\times {10}^{10}$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(+)	(+)	(+)	(+)	(+)
BBPSOjumpC	Best	$6.65\times {10}^6$	$3.80\times {10}^8$	$\mathbf{5}.\mathbf{68}\times {\mathbf{10}}^{\mathbf{3}}$	$1.54\times {10}^2$	$3.98\times {10}^5$	$3.97\times {10}^4$	$1.54\times {10}^9$
	Median	$7.00\times {10}^7$	$6.40\times {10}^8$	$6.49\times {10}^3$	$1.67\times {10}^2$	$4.82\times {10}^5$	$9.33\times {10}^4$	$2.13\times {10}^9$
	Worst	$1.78\times {10}^8$	$9.96\times {10}^8$	$\mathbf{6}.\mathbf{78}\times {\mathbf{10}}^{\mathbf{3}}$	$1.84\times {10}^2$	$5.73\times {10}^5$	$2.87\times {10}^6$	$2.67\times {10}^9$
	Mean	$7.05\times {10}^7$	$6.17\times {10}^8$	$\mathbf{6}.\mathbf{33}\times {\mathbf{10}}^{\mathbf{3}}$	$1.68\times {10}^2$	$4.74\times {10}^5$	$3.39\times {10}^5$	$2.15\times {10}^9$
	Std	$4.81\times {10}^7$	$1.62\times {10}^8$	$3.47\times {10}^2$	$8.39\times {10}^0$	$5.36\times {10}^4$	$7.22\times {10}^5$	$2.95\times {10}^8$
	p-value	$9.46\times {10}^{-2}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(∼)	(+)	(−)	(−)	(+)	(+)	(+)
BBPSOjumpG	Best	$2.91\times {10}^7$	$4.07\times {10}^8$	$5.75\times {10}^3$	$1.54\times {10}^2$	$4.02\times {10}^5$	$3.73\times {10}^4$	$1.59\times {10}^9$
	Median	$8.60\times {10}^7$	$5.41\times {10}^8$	$\mathbf{6}.\mathbf{31}\times {\mathbf{10}}^{\mathbf{3}}$	$1.73\times {10}^2$	$4.74\times {10}^5$	$6.59\times {10}^4$	$2.03\times {10}^9$
	Worst	$1.68\times {10}^8$	$1.01\times {10}^9$	$6.93\times {10}^3$	$1.83\times {10}^2$	$6.44\times {10}^5$	$2.19\times {10}^5$	$2.84\times {10}^9$
	Mean	$7.58\times {10}^7$	$5.75\times {10}^8$	$6.39\times {10}^3$	$1.71\times {10}^2$	$4.89\times {10}^5$	$8.88\times {10}^4$	$2.08\times {10}^9$
	Std	$4.23\times {10}^7$	$1.47\times {10}^8$	$4.19\times {10}^2$	$9.20\times {10}^0$	$7.62\times {10}^4$	$5.95\times {10}^4$	$3.55\times {10}^8$
	p-value	$3.53\times {10}^{-2}$	$6.10\times {10}^{-5}$	$1.83\times {10}^{-4}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(−)	(−)	(+)	(+)	(+)
CLPSO	Best	$6.93\times {10}^6$	$1.31\times {10}^9$	$9.05\times {10}^3$	$\mathbf{8}.\mathbf{89}\times {\mathbf{10}}^{\mathbf{1}}$	$5.57\times {10}^5$	$8.02\times {10}^8$	$2.17\times {10}^9$
	Median	$1.60\times {10}^7$	$1.42\times {10}^9$	$9.30\times {10}^3$	$\mathbf{1}.\mathbf{10}\times {\mathbf{10}}^{\mathbf{2}}$	$5.96\times {10}^5$	$1.02\times {10}^9$	$2.43\times {10}^9$
	Worst	$\mathbf{1}.\mathbf{34}\times {\mathbf{10}}^{\mathbf{8}}$	$1.86\times {10}^9$	$9.67\times {10}^3$	$\mathbf{1}.\mathbf{29}\times {\mathbf{10}}^{\mathbf{2}}$	$6.55\times {10}^5$	$1.78\times {10}^9$	$2.72\times {10}^9$
	Mean	$\mathbf{3}.\mathbf{53}\times {\mathbf{10}}^{\mathbf{7}}$	$1.48\times {10}^9$	$9.32\times {10}^3$	$\mathbf{1}.\mathbf{09}\times {\mathbf{10}}^{\mathbf{2}}$	$6.01\times {10}^5$	$1.10\times {10}^9$	$2.45\times {10}^9$
	Std	$\mathbf{3}.\mathbf{85}\times {\mathbf{10}}^{\mathbf{7}}$	$1.51\times {10}^8$	$\mathbf{1}.\mathbf{58}\times {\mathbf{10}}^{\mathbf{2}}$	$1.23\times {10}^1$	$2.76\times {10}^4$	$2.97\times {10}^8$	$1.49\times {10}^8$
	p-value	$7.62\times {10}^{-1}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(∼)	(+)	(+)	(−)	(+)	(+)	(+)
CPSO-H6	Best	$1.27\times {10}^9$	$1.24\times {10}^{10}$	$1.35\times {10}^4$	$2.22\times {10}^2$	$2.49\times {10}^6$	$7.71\times {10}^{10}$	$1.16\times {10}^{10}$
	Median	$3.46\times {10}^9$	$1.90\times {10}^{10}$	$1.45\times {10}^4$	$2.24\times {10}^2$	$2.90\times {10}^6$	$1.30\times {10}^{11}$	$1.42\times {10}^{10}$
	Worst	$2.26\times {10}^{10}$	$2.78\times {10}^{10}$	$1.58\times {10}^4$	$2.26\times {10}^2$	$3.24\times {10}^6$	$3.02\times {10}^{11}$	$2.08\times {10}^{10}$
	Mean	$5.23\times {10}^9$	$1.88\times {10}^{10}$	$1.45\times {10}^4$	$2.24\times {10}^2$	$2.88\times {10}^6$	$1.55\times {10}^{11}$	$1.54\times {10}^{10}$
	Std	$5.27\times {10}^9$	$3.87\times {10}^9$	$4.92\times {10}^2$	$1.13\times {10}^0$	$1.97\times {10}^5$	$6.96\times {10}^{10}$	$2.74\times {10}^9$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(+)	(+)	(+)	(+)	(+)
HCBBPSO-JC	Best	$\mathbf{1}.\mathbf{06}\times {\mathbf{10}}^{\mathbf{4}}$	$2.51\times {10}^7$	$6.49\times {10}^3$	$2.17\times {10}^2$	$2.36\times {10}^3$	$9.14\times {10}^2$	$\mathbf{7}.\mathbf{99}\times {\mathbf{10}}^{\mathbf{7}}$
	Median	$\mathbf{7}.\mathbf{04}\times {\mathbf{10}}^{\mathbf{6}}$	$\mathbf{2}.\mathbf{96}\times {\mathbf{10}}^{\mathbf{7}}$	$7.15\times {10}^3$	$2.18\times {10}^2$	$3.01\times {10}^3$	$\mathbf{1}.\mathbf{41}\times {\mathbf{10}}^{\mathbf{3}}$	$9.72\times {10}^7$
	Worst	$1.68\times {10}^8$	$\mathbf{3}.\mathbf{20}\times {\mathbf{10}}^{\mathbf{7}}$	$7.62\times {10}^3$	$2.18\times {10}^2$	$3.91\times {10}^3$	$\mathbf{3}.\mathbf{35}\times {\mathbf{10}}^{\mathbf{3}}$	$\mathbf{1}.\mathbf{09}\times {\mathbf{10}}^{\mathbf{8}}$
	Mean	$5.58\times {10}^7$	$\mathbf{2}.\mathbf{96}\times {\mathbf{10}}^{\mathbf{7}}$	$7.15\times {10}^3$	$2.18\times {10}^2$	$3.00\times {10}^3$	$\mathbf{1}.\mathbf{52}\times {\mathbf{10}}^{\mathbf{3}}$	$\mathbf{9}.\mathbf{54}\times {\mathbf{10}}^{\mathbf{7}}$
	Std	$7.00\times {10}^7$	$\mathbf{1}.\mathbf{84}\times {\mathbf{10}}^{\mathbf{6}}$	$3.37\times {10}^2$	$\mathbf{2}.\mathbf{66}\times {\mathbf{10}}^{-\mathbf{1}}$	$4.59\times {10}^2$	$\mathbf{6}.\mathbf{12}\times {\mathbf{10}}^{\mathbf{2}}$	$8.54\times {10}^6$
	p-value	$4.21\times {10}^{-1}$	$5.42\times {10}^{-1}$	$8.04\times {10}^{-1}$	$8.04\times {10}^{-1}$	$9.46\times {10}^{-2}$	$8.90\times {10}^{-1}$	$6.39\times {10}^{-1}$
	(Sign)	(∼)	(∼)	(∼)	(∼)	(∼)	(∼)	(∼)
HCBBPSO-JG	Best	$8.08\times {10}^4$	$\mathbf{2}.\mathbf{08}\times {\mathbf{10}}^{\mathbf{7}}$	$6.59\times {10}^3$	$2.17\times {10}^2$	$\mathbf{2}.\mathbf{24}\times {\mathbf{10}}^{\mathbf{3}}$	$\mathbf{6}.\mathbf{96}\times {\mathbf{10}}^{\mathbf{2}}$	$8.66\times {10}^7$
	Median	$1.68\times {10}^7$	$2.97\times {10}^7$	$7.23\times {10}^3$	$2.18\times {10}^2$	$\mathbf{2}.\mathbf{69}\times {\mathbf{10}}^{\mathbf{3}}$	$1.41\times {10}^3$	$\mathbf{9}.\mathbf{60}\times {\mathbf{10}}^{\mathbf{7}}$
	Worst	$1.58\times {10}^8$	$3.67\times {10}^7$	$7.79\times {10}^3$	$2.18\times {10}^2$	$\mathbf{3}.\mathbf{30}\times {\mathbf{10}}^{\mathbf{3}}$	$3.35\times {10}^3$	$1.10\times {10}^8$
	Mean	$3.97\times {10}^7$	$3.00\times {10}^7$	$7.21\times {10}^3$	$2.18\times {10}^2$	$\mathbf{2}.\mathbf{74}\times {\mathbf{10}}^{\mathbf{3}}$	$1.58\times {10}^3$	$9.65\times {10}^7$
	Std	$5.30\times {10}^7$	$4.15\times {10}^6$	$3.53\times {10}^2$	$3.42\times {10}^{-1}$	$\mathbf{2}.\mathbf{82}\times {\mathbf{10}}^{\mathbf{2}}$	$7.92\times {10}^2$	$\mathbf{6}.\mathbf{88}\times {\mathbf{10}}^{\mathbf{6}}$
Total # of +/∼/−		3/3/0	5/1/0	3/1/2	2/1/3	5/1/0	5/1/0	5/1/0

and

Table 6. A comparison of the results of simulations performed by variants of the PSO algorithm. The best, median, worst, mean, standard deviation, p-value, and significance sign of the 25 runs when the NFEs counter reached $3.0\times {10}^6$ for the functions $f_{15}$∼$f_{20}$ are reported. The p-value was determined using the Wilcoxon signed-rank test between the best algorithm (HCBBPSO-JG) and the others. The significance sign “+” means that the HCBBPSO-JG algorithm was significantly better than the algorithm compared to it, the sign “∼” means that the two were not significantly different, and the sign “−” means that the HCBBPSO-JG algorithm was significantly worse than the algorithm compared to it. The measured value written in bold in colored cells for each function represents the value of the best algorithm for each statistical value.

Algorithm		${\mathit{f}}_{15}$	${\mathit{f}}_{16}$	${\mathit{f}}_{17}$	${\mathit{f}}_{18}$	${\mathit{f}}_{19}$	${\mathit{f}}_{20}$
APSO	Best	$2.39\times {10}^4$	$4.22\times {10}^2$	$1.64\times {10}^6$	$7.02\times {10}^{12}$	$5.93\times {10}^6$	$7.91\times {10}^{12}$
	Median	$2.42\times {10}^4$	$4.28\times {10}^2$	$5.13\times {10}^6$	$7.60\times {10}^{12}$	$7.91\times {10}^6$	$8.37\times {10}^{12}$
	Worst	$2.50\times {10}^4$	$4.30\times {10}^2$	$5.38\times {10}^7$	$7.99\times {10}^{12}$	$2.81\times {10}^7$	$8.54\times {10}^{12}$
	Mean	$2.43\times {10}^4$	$4.28\times {10}^2$	$9.75\times {10}^6$	$7.55\times {10}^{12}$	$1.10\times {10}^7$	$8.34\times {10}^{12}$
	Std	$3.21\times {10}^2$	$1.96\times {10}^0$	$1.45\times {10}^7$	$2.57\times {10}^{11}$	$5.96\times {10}^6$	$1.61\times {10}^{11}$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^5$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(+)	(+)	(+)	(+)
BBPSOjumpC	Best	$7.39\times {10}^3$	$3.45\times {10}^2$	$1.21\times {10}^6$	$4.33\times {10}^8$	$3.10\times {10}^6$	$1.46\times {10}^8$
	Median	$7.95\times {10}^3$	$3.62\times {10}^2$	$1.56\times {10}^6$	$2.39\times {10}^9$	$3.70\times {10}^6$	$9.56\times {10}^8$
	Worst	$8.92\times {10}^3$	$3.77\times {10}^2$	$2.05\times {10}^6$	$1.07\times {10}^{10}$	$4.02\times {10}^6$	$1.49\times {10}^{10}$
	Mean	$8.07\times {10}^3$	$3.62\times {10}^2$	$1.58\times {10}^6$	$3.92\times {10}^9$	$3.72\times {10}^6$	$2.69\times {10}^9$
	Std	$5.43\times {10}^2$	$8.43\times {10}^0$	$2.30\times {10}^5$	$3.26\times {10}^9$	$2.72\times {10}^5$	$3.84\times {10}^9$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(−)	(−)	(+)	(+)	(+)	(+)
BBPSOjumpG	Best	$\mathbf{6}.\mathbf{78}\times {\mathbf{10}}^{\mathbf{3}}$	$3.50\times {10}^2$	$1.27\times {10}^6$	$1.46\times {10}^9$	$3.34\times {10}^6$	$9.52\times {10}^8$
	Median	$\mathbf{7}.\mathbf{89}\times {\mathbf{10}}^{\mathbf{3}}$	$3.67\times {10}^2$	$1.56\times {10}^6$	$3.29\times {10}^9$	$3.66\times {10}^6$	$2.10\times {10}^9$
	Worst	$\mathbf{8}.\mathbf{85}\times {\mathbf{10}}^{\mathbf{3}}$	$3.89\times {10}^2$	$1.86\times {10}^6$	$1.85\times {10}^{10}$	$4.30\times {10}^6$	$3.87\times {10}^{10}$
	Mean	$\mathbf{7}.\mathbf{85}\times {\mathbf{10}}^{\mathbf{3}}$	$3.68\times {10}^2$	$1.58\times {10}^6$	$4.29\times {10}^9$	$3.70\times {10}^6$	$7.95\times {10}^9$
	Std	$5.34\times {10}^2$	$1.19\times {10}^1$	$1.74\times {10}^5$	$4.13\times {10}^9$	$2.27\times {10}^5$	$1.13\times {10}^{10}$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(−)	(−)	(+)	(+)	(+)	(+)
CLPSO	Best	$1.36\times {10}^4$	$\mathbf{2}.\mathbf{64}\times {\mathbf{10}}^{\mathbf{2}}$	$1.20\times {10}^6$	$3.32\times {10}^{10}$	$5.39\times {10}^6$	$4.55\times {10}^{10}$
	Median	$1.37\times {10}^4$	$\mathbf{2}.\mathbf{98}\times {\mathbf{10}}^{\mathbf{2}}$	$1.41\times {10}^6$	$3.92\times {10}^{10}$	$5.85\times {10}^6$	$5.61\times {10}^{10}$
	Worst	$1.39\times {10}^4$	$\mathbf{3}.\mathbf{10}\times {\mathbf{10}}^{\mathbf{2}}$	$1.55\times {10}^6$	$4.54\times {10}^{10}$	$6.54\times {10}^6$	$6.16\times {10}^{10}$
	Mean	$1.37\times {10}^4$	$\mathbf{2}.\mathbf{95}\times {\mathbf{10}}^{\mathbf{2}}$	$1.39\times {10}^6$	$3.90\times {10}^{10}$	$5.86\times {10}^6$	$5.54\times {10}^{10}$
	Std	$\mathbf{9}.\mathbf{77}\times {\mathbf{10}}^{\mathbf{1}}$	$1.28\times {10}^1$	$9.71\times {10}^4$	$4.14\times {10}^9$	$3.12\times {10}^5$	$4.51\times {10}^9$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(−)	(+)	(+)	(+)	(+)
CPSO-H6	Best	$1.34\times {10}^4$	$4.06\times {10}^2$	$2.86\times {10}^6$	$1.52\times {10}^{12}$	$1.00\times {10}^7$	$1.58\times {10}^{12}$
	Median	$1.41\times {10}^4$	$4.08\times {10}^2$	$3.20\times {10}^6$	$1.78\times {10}^{12}$	$1.33\times {10}^7$	$1.94\times {10}^{12}$
	Worst	$1.49\times {10}^4$	$4.12\times {10}^2$	$3.77\times {10}^6$	$2.12\times {10}^{12}$	$2.30\times {10}^7$	$2.35\times {10}^{12}$
	Mean	$1.41\times {10}^4$	$4.09\times {10}^2$	$3.23\times {10}^6$	$1.76\times {10}^{12}$	$1.40\times {10}^7$	$1.98\times {10}^{12}$
	Std	$4.21\times {10}^2$	$1.89\times {10}^0$	$2.70\times {10}^5$	$1.56\times {10}^{11}$	$4.03\times {10}^6$	$1.97\times {10}^{11}$
	p-value	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$	$6.10\times {10}^{-5}$
	(Sign)	(+)	(+)	(+)	(+)	(+)	(+)
HCBBPSO-JC	Best	$1.07\times {10}^4$	$3.96\times {10}^2$	$1.59\times {10}^4$	$2.24\times {10}^3$	$\mathbf{4}.\mathbf{50}\times {\mathbf{10}}^{\mathbf{5}}$	$1.49\times {10}^3$
	Median	$1.16\times {10}^4$	$3.97\times {10}^2$	$2.10\times {10}^4$	$3.91\times {10}^3$	$7.92\times {10}^5$	$\mathbf{1}.\mathbf{69}\times {\mathbf{10}}^{\mathbf{3}}$
	Worst	$1.20\times {10}^4$	$3.97\times {10}^2$	$2.38\times {10}^4$	$\mathbf{5}.\mathbf{65}\times {\mathbf{10}}^{\mathbf{3}}$	$\mathbf{9}.\mathbf{25}\times {\mathbf{10}}^{\mathbf{5}}$	$2.07\times {10}^3$
	Mean	$1.15\times {10}^4$	$3.97\times {10}^2$	$2.07\times {10}^4$	$4.00\times {10}^3$	$7.71\times {10}^5$	$1.74\times {10}^3$
	Std	$4.30\times {10}^2$	$3.97\times {10}^{-1}$	$2.37\times {10}^3$	$\mathbf{8}.\mathbf{94}\times {\mathbf{10}}^{\mathbf{2}}$	$\mathbf{1}.\mathbf{59}\times {\mathbf{10}}^{\mathbf{5}}$	$1.64\times {10}^2$
	p-value	$7.20\times {10}^{-1}$	$3.02\times {10}^{-2}$	$4.27\times {10}^{-4}$	$4.46\times {10}^{-1}$	$1.00\times {10}^0$	$1.88\times {10}^{-1}$
	(Sign)	(∼)	(+)	(+)	(∼)	(∼)	(∼)
HCBBPSO-JG	Best	$1.06\times {10}^4$	$3.96\times {10}^2$	$\mathbf{1}.\mathbf{37}\times {\mathbf{10}}^{\mathbf{4}}$	$\mathbf{1}.\mathbf{90}\times {\mathbf{10}}^{\mathbf{3}}$	$5.29\times {10}^5$	$\mathbf{1}.\mathbf{40}\times {\mathbf{10}}^{\mathbf{3}}$
	Median	$1.16\times {10}^4$	$3.96\times {10}^2$	$\mathbf{1}.\mathbf{77}\times {\mathbf{10}}^{\mathbf{4}}$	$\mathbf{3}.\mathbf{69}\times {\mathbf{10}}^{\mathbf{3}}$	$\mathbf{7}.\mathbf{14}\times {\mathbf{10}}^{\mathbf{5}}$	$1.70\times {10}^3$
	Worst	$1.23\times {10}^4$	$3.97\times {10}^2$	$\mathbf{2}.\mathbf{12}\times {\mathbf{10}}^{\mathbf{4}}$	$6.46\times {10}^3$	$1.06\times {10}^6$	$\mathbf{1}.\mathbf{92}\times {\mathbf{10}}^{\mathbf{3}}$
	Mean	$1.15\times {10}^4$	$3.96\times {10}^2$	$\mathbf{1}.\mathbf{77}\times {\mathbf{10}}^{\mathbf{4}}$	$\mathbf{3}.\mathbf{61}\times {\mathbf{10}}^{\mathbf{3}}$	$\mathbf{7}.\mathbf{60}\times {\mathbf{10}}^{\mathbf{5}}$	$\mathbf{1}.\mathbf{69}\times {\mathbf{10}}^{\mathbf{3}}$
	Std	$4.93\times {10}^2$	$\mathbf{3}.\mathbf{87}\times {\mathbf{10}}^{-\mathbf{1}}$	$\mathbf{1}.\mathbf{97}\times {\mathbf{10}}^{\mathbf{3}}$	$1.28\times {10}^3$	$1.62\times {10}^5$	$\mathbf{1}.\mathbf{57}\times {\mathbf{10}}^{\mathbf{2}}$
Total # of +/∼/−		3/2/1	3/0/3	6/0/0	5/1/0	5/1/0	5/1/0

provide the results of the 25 runs of each variant of the PSO algorithm used in the comparison process when the NFEs counter reached $=3.0\times {10}^6$ for the 20 functions. We conducted several statistical hypothesis tests on these results. First, we performed the Friedman rank test on the data from all of the algorithms. Next, if there was a significant difference at the $5\%$ significance level, we performed the Wilcoxon signed-rank test on the best algorithm, i.e., the HCBBPSO-JG algorithm, as well as the other algorithms and marked the results of each statistical significance test with its p-value and sign in the tables; the sign “+” means that the HCBBPSO-JG algorithm was significantly better than the algorithm compared to it, the sign “∼” means that the two were not significantly different, and the sign “−” means that the HCBBPSO-JG algorithm was significantly worse than the algorithm compared to it. The results of the Friedman rank test show that there were significant differences in the data from all of the algorithms at the $p\text{-}\mathrm{value}\approx 0\ll 0.05$ significance level; therefore, the Wilcoxon signed-rank test was performed on the HCBBPSO-JG algorithm and the other algorithms. In the tables, the measured value of each function written in bold in colored cells represents the value of the best algorithm for each statistical value.

In these comparative results, the proposed HCBBPSO-Jx algorithms performed better than the other algorithms; they won for a total of 12 functions in terms of the mean NFEs, which is similar to the results obtained using the CEC 2010 LSGO Challenge scoring system. In particular, the HCBBPSO-JG algorithm stayed ahead of the HCBBPSO-JC algorithm with significant differences in the three functions $f_7$, $f_{16}$, and $f_{17}$. However, the HCBBPSO-Jx algorithms were the weakest of the variants of Rastrigin’s function, i.e., $f_2$, $f_5$, $f_{10}$, and $f_{15}$. The cause of this phenomenon can be understood by examining the convergence curves shown in Figure 3. We know from the curves for $f_2$, $f_5$, $f_{10}$, and $f_{15}$ that the HCBBPSO-JG algorithm prematurely converged to a local optimum only for the functions that were connected to Rastrigin’s function. The first cause for this result was to simulate all algorithms using the same total number of particles in order to ensure fairness in comparison simulation. The total number of particles directly affects the increase or decrease of NFEs (Number of Function Evaluations) of an algorithm because one particle must use at least one function evaluation to be evaluated for the solution it finds itself. Therefore, it cannot be a fair comparison if simulations are performed with different particle size. The second reason for these results is that the optimal parameter value for the parameter split factor K, which governs the performance of the HCBBPSO-Jx algorithms, was not used. Selecting this parameter K is very hard because it depends on the problems that want to be solved. In particular, the problems to solve with the algorithms proposed in this paper are high-dimensional problems with high complexity. Therefore, the issue of selecting optimal parameter values for the proposed HCBBPSO-Jx algorithms is to be left behind for further in-depth research later as the further works of this paper.

4.3. The Results of Simulations Using the HCBBPSO-JG Algorithm for the CEC 2010 Benchmark Functions

Table 7. The results of simulations using the best algorithm, the HCBBPSO-JG algorithm: the best, median, worst, mean, and standard deviation of the 25 runs when the NFEs counter reached $1.2\times {10}^5$, $6.0\times {10}^5$, and $3.0\times {10}^6$ are reported.

1000D		${\mathit{f}}_1$	${\mathit{f}}_2$	${\mathit{f}}_3$	${\mathit{f}}_4$	${\mathit{f}}_5$	${\mathit{f}}_6$	${\mathit{f}}_7$
$\mathbf{1}.\mathbf{2}\times {\mathbf{10}}^{\mathbf{5}}$	Best	$7.15\times {10}^8$	$3.47\times {10}^3$	$7.28\times {10}^0$	$7.35\times {10}^{12}$	$4.21\times {10}^8$	$1.95\times {10}^7$	$7.55\times {10}^9$
	Median	$1.36\times {10}^9$	$3.84\times {10}^3$	$8.34\times {10}^0$	$1.78\times {10}^{13}$	$5.58\times {10}^8$	$1.98\times {10}^7$	$1.06\times {10}^{10}$
	Worst	$2.35\times {10}^9$	$3.98\times {10}^3$	$9.12\times {10}^0$	$2.17\times {10}^{13}$	$7.34\times {10}^8$	$1.99\times {10}^7$	$1.76\times {10}^{10}$
	Mean	$1.33\times {10}^9$	$3.79\times {10}^3$	$8.26\times {10}^0$	$1.58\times {10}^{13}$	$5.57\times {10}^8$	$1.97\times {10}^7$	$1.11\times {10}^{10}$
	Std	$3.87\times {10}^8$	$1.48\times {10}^2$	$5.62\times {10}^{-1}$	$5.12\times {10}^{12}$	$1.04\times {10}^8$	$9.64\times {10}^4$	$2.84\times {10}^9$
$\mathbf{6}.\mathbf{0}\times {\mathbf{10}}^{\mathbf{5}}$	Best	$5.69\times {10}^1$	$2.51\times {10}^3$	$2.08\times {10}^0$	$1.32\times {10}^{12}$	$4.21\times {10}^8$	$1.95\times {10}^7$	$1.38\times {10}^8$
	Median	$2.03\times {10}^2$	$2.80\times {10}^3$	$3.40\times {10}^0$	$3.61\times {10}^{12}$	$5.58\times {10}^8$	$1.98\times {10}^7$	$2.89\times {10}^8$
	Worst	$3.94\times {10}^3$	$3.04\times {10}^3$	$5.18\times {10}^0$	$5.66\times {10}^{12}$	$7.34\times {10}^8$	$1.99\times {10}^7$	$5.39\times {10}^8$
	Mean	$4.44\times {10}^2$	$2.78\times {10}^3$	$3.47\times {10}^0$	$3.67\times {10}^{12}$	$5.57\times {10}^8$	$1.97\times {10}^7$	$3.00\times {10}^8$
	Std	$9.77\times {10}^2$	$1.62\times {10}^2$	$8.55\times {10}^{-1}$	$1.38\times {10}^{12}$	$1.04\times {10}^8$	$9.79\times {10}^4$	$1.11\times {10}^8$
$\mathbf{3}.\mathbf{0}\times {\mathbf{10}}^{\mathbf{6}}$	Best	$4.16\times {10}^{-20}$	$2.51\times {10}^3$	$2.08\times {10}^0$	$2.93\times {10}^{11}$	$4.21\times {10}^8$	$1.95\times {10}^7$	$3.71\times {10}^4$
	Median	$7.76\times {10}^{-19}$	$2.80\times {10}^3$	$3.40\times {10}^0$	$8.84\times {10}^{11}$	$5.58\times {10}^8$	$1.98\times {10}^7$	$1.66\times {10}^5$
	Worst	$2.63\times {10}^{-16}$	$3.04\times {10}^3$	$5.18\times {10}^0$	$1.38\times {10}^{12}$	$7.30\times {10}^8$	$1.99\times {10}^7$	$2.45\times {10}^5$
	Mean	$2.81\times {10}^{-17}$	$2.77\times {10}^3$	$3.47\times {10}^0$	$7.82\times {10}^{11}$	$5.54\times {10}^8$	$1.97\times {10}^7$	$1.45\times {10}^5$
	Std	$6.88\times {10}^{-17}$	$1.63\times {10}^2$	$8.55\times {10}^{-1}$	$3.23\times {10}^{11}$	$9.93\times {10}^7$	$1.08\times {10}^5$	$7.32\times {10}^4$
1000D		${\mathit{f}}_{\mathbf{8}}$	${\mathit{f}}_{\mathbf{9}}$	${\mathit{f}}_{\mathbf{10}}$	${\mathit{f}}_{\mathbf{11}}$	${\mathit{f}}_{\mathbf{12}}$	${\mathit{f}}_{\mathbf{13}}$	${\mathit{f}}_{\mathbf{14}}$
$\mathbf{1}.\mathbf{2}\times {\mathbf{10}}^{\mathbf{5}}$	Best	$6.32\times {10}^7$	$9.32\times {10}^8$	$7.64\times {10}^3$	$2.18\times {10}^2$	$7.09\times {10}^5$	$2.04\times {10}^6$	$2.40\times {10}^9$
	Median	$1.63\times {10}^8$	$1.10\times {10}^9$	$8.38\times {10}^3$	$2.19\times {10}^2$	$8.67\times {10}^5$	$3.96\times {10}^6$	$2.75\times {10}^9$
	Worst	$7.10\times {10}^9$	$1.60\times {10}^9$	$8.85\times {10}^3$	$2.20\times {10}^2$	$9.79\times {10}^5$	$7.47\times {10}^6$	$3.09\times {10}^9$
	Mean	$1.24\times {10}^9$	$1.13\times {10}^9$	$8.25\times {10}^3$	$2.19\times {10}^2$	$8.80\times {10}^5$	$4.05\times {10}^6$	$2.76\times {10}^9$
	Std	$2.38\times {10}^9$	$1.51\times {10}^8$	$3.42\times {10}^2$	$3.85\times {10}^{-1}$	$7.55\times {10}^4$	$1.77\times {10}^6$	$2.31\times {10}^8$
$\mathbf{6}.\mathbf{0}\times {\mathbf{10}}^{\mathbf{5}}$	Best	$1.29\times {10}^6$	$1.24\times {10}^8$	$6.59\times {10}^3$	$2.17\times {10}^2$	$1.50\times {10}^5$	$1.67\times {10}^3$	$4.86\times {10}^8$
	Median	$6.85\times {10}^7$	$1.65\times {10}^8$	$7.27\times {10}^3$	$2.18\times {10}^2$	$1.72\times {10}^5$	$5.39\times {10}^3$	$5.37\times {10}^8$
	Worst	$2.26\times {10}^8$	$1.98\times {10}^8$	$7.79\times {10}^3$	$2.18\times {10}^2$	$1.96\times {10}^5$	$1.54\times {10}^4$	$6.23\times {10}^8$
	Mean	$7.48\times {10}^7$	$1.64\times {10}^8$	$7.21\times {10}^3$	$2.18\times {10}^2$	$1.72\times {10}^5$	$5.76\times {10}^3$	$5.41\times {10}^8$
	Std	$6.83\times {10}^7$	$1.82\times {10}^7$	$3.53\times {10}^2$	$3.41\times {10}^{-1}$	$1.44\times {10}^4$	$4.08\times {10}^3$	$4.43\times {10}^7$
$\mathbf{3}.\mathbf{0}\times {\mathbf{10}}^{\mathbf{6}}$	Best	$8.08\times {10}^4$	$2.08\times {10}^7$	$6.59\times {10}^3$	$2.17\times {10}^2$	$2.24\times {10}^3$	$6.96\times {10}^2$	$8.66\times {10}^7$
	Median	$1.68\times {10}^7$	$2.97\times {10}^7$	$7.23\times {10}^3$	$2.18\times {10}^2$	$2.69\times {10}^3$	$1.41\times {10}^3$	$9.60\times {10}^7$
	Worst	$1.58\times {10}^8$	$3.67\times {10}^7$	$7.79\times {10}^3$	$2.18\times {10}^2$	$3.30\times {10}^3$	$3.35\times {10}^3$	$1.10\times {10}^8$
	Mean	$3.97\times {10}^7$	$3.00\times {10}^7$	$7.21\times {10}^3$	$2.18\times {10}^2$	$2.74\times {10}^3$	$1.58\times {10}^3$	$9.65\times {10}^7$
	Std	$5.30\times {10}^7$	$4.15\times {10}^6$	$3.53\times {10}^2$	$3.42\times {10}^{-1}$	$2.82\times {10}^2$	$7.92\times {10}^2$	$6.88\times {10}^6$
1000D		${\mathit{f}}_{\mathbf{15}}$	${\mathit{f}}_{\mathbf{16}}$	${\mathit{f}}_{\mathbf{17}}$	${\mathit{f}}_{\mathbf{18}}$	${\mathit{f}}_{\mathbf{19}}$	${\mathit{f}}_{\mathbf{20}}$
$\mathbf{1}.\mathbf{2}\times {\mathbf{10}}^{\mathbf{5}}$	Best	$1.11\times {10}^4$	$4.00\times {10}^2$	$1.65\times {10}^6$	$8.22\times {10}^8$	$5.37\times {10}^6$	$8.05\times {10}^8$
	Median	$1.21\times {10}^4$	$4.01\times {10}^2$	$1.82\times {10}^6$	$1.12\times {10}^9$	$7.16\times {10}^6$	$1.05\times {10}^9$
	Worst	$1.28\times {10}^4$	$4.02\times {10}^2$	$2.03\times {10}^6$	$1.73\times {10}^9$	$8.44\times {10}^6$	$1.76\times {10}^9$
	Mean	$1.20\times {10}^4$	$4.01\times {10}^2$	$1.84\times {10}^6$	$1.13\times {10}^9$	$7.02\times {10}^6$	$1.14\times {10}^9$
	Std	$5.03\times {10}^2$	$7.48\times {10}^{-1}$	$1.16\times {10}^5$	$2.86\times {10}^8$	$8.69\times {10}^5$	$2.39\times {10}^8$
$\mathbf{6}.\mathbf{0}\times {\mathbf{10}}^{\mathbf{5}}$	Best	$1.06\times {10}^4$	$3.96\times {10}^2$	$4.02\times {10}^5$	$2.13\times {10}^4$	$2.40\times {10}^6$	$3.14\times {10}^3$
	Median	$1.16\times {10}^4$	$3.96\times {10}^2$	$4.69\times {10}^5$	$3.98\times {10}^4$	$2.79\times {10}^6$	$3.55\times {10}^3$
	Worst	$1.23\times {10}^4$	$3.97\times {10}^2$	$5.25\times {10}^5$	$6.12\times {10}^4$	$3.55\times {10}^6$	$4.52\times {10}^3$
	Mean	$1.15\times {10}^4$	$3.96\times {10}^2$	$4.66\times {10}^5$	$4.10\times {10}^4$	$2.86\times {10}^6$	$3.63\times {10}^3$
	Std	$4.93\times {10}^2$	$3.87\times {10}^{-1}$	$3.38\times {10}^4$	$1.17\times {10}^4$	$3.39\times {10}^5$	$3.71\times {10}^2$
$\mathbf{3}.\mathbf{0}\times {\mathbf{10}}^{\mathbf{6}}$	Best	$1.06\times {10}^4$	$3.96\times {10}^2$	$1.37\times {10}^4$	$1.90\times {10}^3$	$5.29\times {10}^5$	$1.40\times {10}^3$
	Median	$1.16\times {10}^4$	$3.96\times {10}^2$	$1.77\times {10}^4$	$3.69\times {10}^3$	$7.14\times {10}^5$	$1.70\times {10}^3$
	Worst	$1.23\times {10}^4$	$3.97\times {10}^2$	$2.12\times {10}^4$	$6.46\times {10}^3$	$1.06\times {10}^6$	$1.92\times {10}^3$
	Mean	$1.15\times {10}^4$	$3.96\times {10}^2$	$1.77\times {10}^4$	$3.61\times {10}^3$	$7.60\times {10}^5$	$1.69\times {10}^3$
	Std	$4.93\times {10}^2$	$3.87\times {10}^{-1}$	$1.97\times {10}^3$	$1.28\times {10}^3$	$1.62\times {10}^5$	$1.57\times {10}^2$

include the simulation results of the best algorithm, the HCBBPSO-JG algorithm. The best, median, worst, mean, and standard deviation of the 25 runs when the NFEs counter reached $1.2$ × ${10}^5,6.0\times {10}^5$, and $3.0\times {10}^6$ are reported. This information ws used in the CEC 2010 LSGO Challenge scoring system, and the CEC 2010 LSGO Challenge included a rule stating that this information must be opened with the single convergence curves for several functions that were specified in advance. Figure 3 show the convergence curves for the designated functions, $f_2$, $f_5$, $f_8$, $f_{10}$, $f_{13}$, $f_{15}$, $f_{18}$, and $f_{20}$. For each function, the curve has been graphed using the average of the results of all 25 runs. As mentioned previously, the HCBBPSO-JG algorithm converged prematurely for the functions $f_2$, $f_5$, $f_{10}$, and $f_{15}$, which were related to Rastrigin’s function. For the other functions, there were improvements until the maximum NFEs was reached, which helped the HCBBPSO-JG algorithm perform well.

5. Conclusions

This paper proposed heterogeneous cooperative BBPSO algorithms that used a jumping strategy, which was strengthened for use with high-dimensional optimization problems by combining an improved exploration ability, which was introduced by means of heterogeneous cooperation and the jumping strategy, with the merits of BBPSO, which is simple but robust because it does not need to consider the selection of controllable parameters for the PSO algorithm and because its performance is not affected by the values of these parameters.

In the comparative simulations based on the 20 qualified benchmark functions and the evaluation system used in the CEC 2010 LSGO Challenge, the HCBBPSO-Jx algorithms provided improved results for most of the functions; notably, the HCBBPSO-JG algorithm performed the best according to the overall evaluation criteria. Although the proposed algorithms converged prematurely for several benchmark functions that were related to Rastrigin’s function, they will be improved in future work. Therefore, the results of this study lead us to conclude that the proposed HCBBPSO-JG algorithm is useful as an optimizer for solving high-dimensional problems. In future work, the proposed algorithm will be improved for Rastrigin’s function, tested with additional benchmark functions and compared with other state-of-the-art optimizers. We will also study the parameter split factor K, because it is very hard to select K depending on the problem. In particular, the problems to solve with the algorithms proposed in this paper are high-dimensional problems with high complexity. In addition, the further study will also be conducted on other parameters to improve the performance of the proposed algorithm.