Elite Directed Particle Swarm Optimization with Historical Information for High-Dimensional Problems

Abstract

High-dimensional optimization problems are ubiquitous in every field nowadays, which seriously challenge the optimization ability of existing optimizers. To solve this kind of optimization problems effectively, this paper proposes an elite-directed particle swarm optimization (EDPSO) with historical information to explore and exploit the high-dimensional solution space efficiently. Specifically, in EDPSO, the swarm is first separated into two exclusive sets based on the Pareto principle (80-20 rule), namely the elite set containing the top best 20% of particles and the non-elite set consisting of the remaining 80% of particles. Then, the non-elite set is further separated into two layers with the same size from the best to the worst. As a result, the swarm is divided into three layers. Subsequently, particles in the third layer learn from those in the first two layers, while particles in the second layer learn from those in the first layer, on the condition that particles in the first layer remain unchanged. In this way, the learning effectiveness and the learning diversity of particles could be largely promoted. To further enhance the learning diversity of particles, we maintain an additional archive to store obsolete elites, and use the predominant elites in the archive along with particles in the first two layers to direct the update of particles in the third layer. With these two mechanisms, the proposed EDPSO is expected to compromise search intensification and diversification well at the swarm level and the particle level, to explore and exploit the solution space. Extensive experiments are conducted on the widely used CEC’2010 and CEC’2013 high-dimensional benchmark problem sets to validate the effectiveness of the proposed EDPSO. Compared with several state-of-the-art large-scale algorithms, EDPSO is demonstrated to achieve highly competitive or even much better performance in tackling high-dimensional problems.

1. Introduction

With the rapid development of big data and the Internet of Things (IoT), the dimensionality of optimization problems is becoming increasingly higher and higher [1,2], leading to the emergence of high-dimensional optimization problems [3,4,5]. Such a kind of optimization problem brings many undesirable challenges to existing optimizers. Specifically, with the dimensionality increasing, the solution space of optimization problems grows exponentially, resulting in the search effectiveness and the efficiency of existing optimizers degrading dramatically [6]. On the other hand, wide and flat local regions are also likely to increase rapidly as the dimensionality grows, especially for multimodal problems. This results in it being hard for existing optimizers to effectively escape from local areas, such that global optima cannot be located efficiently [7,8].

To address the above challenges, a lot of metaheuristic algorithms have been proposed by taking inspiration from nature and other disciplines. For example, imitating the laws of physics, physics-based optimization algorithms such as Henry gas solubility optimization (HGSO) [9], and artificial electric field algorithm (AEFA) [10], have been developed. Inspired by the social behaviors of animals, swarm intelligence optimization algorithms, such as particle swarm optimization (PSO) [11,12], ant colony optimization (ACO) [13,14,15], salp swarm algorithm (SSA) [16] and grey wolf optimization (GWO) [17], have been devised. Derived from competitive behaviors in sport playing, sport inspiration algorithms, such as the most valuable player algorithm (MVPA) [18] have been designed. Arisen from theories in mathematics, mathematics inspiration optimization, such as arithmetic optimization algorithm (AOA) [19] and sine cosine algorithm (SCA) [20], have been proposed. Inspired by the thoughts of species evolution, evolutionary algorithms, such as genetic algorithms (GA) [21] and differential evolution algorithms (DE) [22,23], have been developed. In addition, some other metaheuristic algorithms, such as adaptive hybrid approach (AHA) [24], Harris hawks optimization (HHO) [25], hunger games search (HGS) [26] and golden jackal optimization (GJO) [27], have also been designed to solve optimization problems.

Among the above different kinds of metaheuristic algorithms, PSO has been researched the most in dealing with large-scale optimization [28,29,30,31,32]. In a broad sense, existing PSO research for high-dimensional optimization problems can be categorized into two main directions [33]: cooperative co-evolutionary PSOs (CCPSOs) [34,35,36] and holistic large-scale PSOs [37,38,39,40,41].

CCPSOs [42,43] first adopt the divide-and-conquer technique to divide a high-dimensional problem into a number of low-dimensional sub-problems, and then utilize traditional PSOs for low-dimensional problems to separately solve the decomposed sub-problems. Since Potter [44] proposed the cooperative co-evolutionary (CC) framework, researchers have employed this framework into different evolutionary algorithms and developed many cooperative co-evolutionary algorithms (CCEAs) [43,45], among which CCPSOs [35,46] have been developed by introducing PSOs into the CC framework.

However, the optimization performance of CCEAs including CCPSOs heavily relies on the decomposition accuracy in dividing the high-dimensional problem into sub-problems, since the optimization of interacting variables usually interferes with each other [43]. Ideally, a good decomposition strategy should place interacting variables into the same sub-problem, so that they can be optimized together. Nevertheless, without prior knowledge on the correlations between variables, it is hard to accurately decompose a high-dimensional problem into sub-problems. To solve this issue, researchers have been devoted to designing effective decomposition strategies to divide a high-dimensional problem into sub-problems as accurately as possible by detecting the correlations between variables [47]. As a consequence, a lot of remarkable decomposition strategies [48,49,50,51,52] have been devised and assisted CCEAs, including CCPSOs, to achieve very promising performance in tackling large-scale optimization.

Different from CCPSOs, holistic large-scale PSOs [33,53] still consider optimizing all variables together such as traditional PSOs. The key role in designing effective holistic large-scale PSOs lies in devising effective learning strategies for PSO to improve the search diversity, largely so that particles in the swarm could search the solution space in different directions and locate the promising areas fast. To achieve this goal, researchers have abandoned historical information, such as the personal best positions pbests, the global best position gbest, or the neighbor best position nbest, and attempted to employ predominate particles in the current swarm to guide the learning of poor particles. Along this line, many novel effective learning strategies [29,31,32,54,55] have been proposed, such as the competitive learning strategy [29], the level-based learning strategy [54], the social learning strategy [56] and the granularity learning strategy [39], etc.

Though the above large-scale PSO variants have shown promising performance in certain kinds of high-dimensional problems, they are still confronted with many limitations in solving complicated large-scale optimization problems, like those with overlapping interacting variables and those with many wide and flat local basins. For CCPSOs, though, the advanced decomposition strategies could assist them to achieve promising performance. Confronted with problems with overlapping interacting variables, they would place many variables into the same sub-problem, leading to several large-scale sub-problems with a lot of interacting variables, or even place all variables into one same group. In this situation, traditional PSOs designed for low-dimensional problems, lose their effectiveness in solving these large-scale sub-problems. As for holistic PSOs, they are still confronted with premature convergence when solving complex problems with many wide and flat local regions. As a consequence, how to improve the optimization ability of PSO in solving large-scale optimization problems still deserves careful research.

To the above end, this paper proposes an elite-directed particle swarm optimization with historical information (EDPSO) to tackle large-scale optimization by taking inspiration from the “Pareto Principle” [57], which is also known as the 80-20 rule that 80% of the consequences come from 20% of the causes, asserting an unequal relationship between inputs and outputs. Specifically, the main components of the proposed EDPSO are summarized as follows.

(1)

An elite-directed learning strategy is devised to let elite particles in the current swarm direct the update of non-elite particles. Specifically, particles in the current swarm are first divided into two separate sets: the elite set consisting of the top 20% best particles and the non-elite set containing the remaining 80% of particles. Then, the non-elite set is further equally separated into two layers. In this way, actually, the swarm is divided into three layers: the elite layer, the better half of the non-elite set, and the worse half of the non-elite set. Subsequently, particles in the last layer are updated by learning from those in the first two layers, and the ones in the second layers are guided by those in the first layers with particles in the first layer (namely the elite layer) unchanged and directly entering the next generation. In this manner, particles in different layers are updated in different ways. Therefore, the learning diversity and the learning effectiveness of particles are expectedly largely improved.

(2)

An additional archive is maintained to store the obsolete elites in the elite set and then is used to cooperate with particles in the first two layers to direct the update of non-elite particles, so that the diversity of guiding exemplars could be promoted and thus the learning diversity of particles is further enhanced.

With the above two techniques, the proposed EDPSO is expected to compromise the search diversity and fast convergence well to explore and exploit the high-dimensional space appropriately to find high-quality solutions. To verify the effectiveness of the proposed EDPSO, extensive experiments are conducted to compare it with several state-of-the-art large-scale optimizers on the widely used CEC’2010 [58] and CEC’2013 [59] large-scale benchmark optimization problem sets. Meanwhile, deep investigations on EDPSO are also conducted to find what contributes to its promising performance.

The rest of this paper is organized as follows. Section 2 reviews closely related work on the classical PSO and large-scale PSO variants. In Section 3, the proposed EDPSO is presented in detail. Subsequently, extensive experiments are performed to compare the proposed algorithm with state-of-the-art peer large-scale optimizers in Section 4. Lastly, we end this paper with the conclusion shown in Section 5.

2. Related Work

Without loss of generality, this paper considers the following minimization problem:

$$\min f(\mathit{x}),\mathit{x}\in R^D$$

(1)

where D is the dimension size, and x is the variable vector that needs to be optimized. In this paper, the function value is taken as the fitness value of a particle.

2.1. Canonical PSO

In the classical PSO [60], each particle is denoted by two vectors, namely the position vector x and the velocity vector v. Then, each particle is updated by cognitively learning from its own personal best position pbest, and socially learning from the global best position gbest of the whole swarm. Specifically, each particle is updated as follows:

$$v_i^d\leftarrow \mathsf{\omega }\ast v_i^d+c_1\ast r_1^d\ast (pbes{t}_i^d-x_i^{\mathrm{d}})+c_2\ast r_2^d\ast (gbes{t}^d-x_i^{\mathrm{d}})$$

(2)

$$x_i^d\leftarrow x_i^d+v_i^d$$

(3)

where ${\mathit{x}}_i=\left[x_i^1,x_i^2,x_i^3,\dots ,x_i^D\right]$ and ${\mathit{v}}_i=\left[v_i^1,v_i^2,v_i^3,\dots ,v_i^D\right]$ are the position vector and the velocity vector of the ith particle. $\mathit{p}\mathit{b}\mathit{e}\mathit{s}{\mathit{t}}_{\mathit{i}}=\left[pbes{t}_i^1,pbes{t}_i^2,pbes{t}_i^3,\dots ,pbes{t}_i^D\right]$ is the personal best position of the ith particle, while $\mathit{g}\mathit{b}\mathit{e}\mathit{s}\mathit{t}=\left[gbes{t}^1,gbes{t}^2,gbes{t}^3,\dots ,gbes{t}^D\right]$ is the global best position found so far by the swarm. As for the parameters, w is termed as the inertia weight, which is used to keep the velocity of the ith particle in the previous generation. $c_1$ and $c_2$ are two acceleration coefficients, and $r_1$ and $r_2$ are two real random numbers uniformly sampled within [0, 1].

Kennedy and Eberhart [60] have taken the second part and the third part in the right of Equation (2) as the cognitive component and the social component, respectively. Since gbest is the global best position of the whole swarm and is shared by all particles, the learning diversity of particles is very limited in classical PSO. In addition, due to the greedy attraction of gbest, the classical PSO encounters premature convergence when dealing with multimodal problems [61,62].

To improve the optimization performance of PSO in coping with multimodal problems, in the early stage, some local topologies have been developed to select a less greedy guiding exemplar for each particle to replace gbest, so that PSO could be effective to escape from local basins [63], such as the ring topology [64], the random topology [65], the star topology [66], and the island-model topology [67]. Later, researchers have developed a lot of novel effective learning strategies [62] to improve the learning effectiveness of particles, such as the comprehensive learning strategy [62], the orthogonal learning strategy [61], etc.

Although a lot of PSO variants have been developed, most of them are specially designed for low-dimensional problems. When dealing with high-dimensional problems, their optimization performance deteriorates drastically [6,68], which usually results from “the curse of dimensionality” [69].

2.2. Large Scale PSOs

To effectively solve large-scale optimization problems, researchers have been devoted to especially designing effective updating strategies suitable for high-dimensional problems for PSO. As a result, an ocean of remarkable large-scale PSO variants have emerged [29,31,32,46,70]. Broadly speaking, existing large-scale PSO variants can be roughly classified into two main categories [7,8], namely cooperative co-evolutionary PSOs (CCPSOs) [34,35,36,46,70] and holistic large-scale PSOs [31,32,54,55,71].

2.2.1. Cooperative Co-Evolutionary PSOs

Since the development of the cooperative co-evolutionary (CC) framework [72], many researchers have introduced traditional PSOs for low-dimensional problems into the CC framework, leading to CCPSOs. In [46], the first CCPSO was proposed by first randomly dividing a D-dimensional problem into K sub-problems with each containing D/K variables and then employing the classical PSO to optimize each sub-problem separately. In [46], an improved CCPSO, named CCPSO-H_K was developed by hybridizing the classical PSO and CCPSO alternatively to optimize a high-dimensional problem. Specifically, after CCPSO is executed, the classical PSO is performed in the next generation, and vice versa. To alleviate the sensitivity to the parameter K, another improved version of CCPSO, named CCPSO2 [35], was devised by using an integer pool consisting of different group numbers. Specifically, this algorithm first randomly selects a group number from the pool every time gbest is not improved and then divides the high-dimensional problem randomly into sub-problems based on the selected number. In [70], a cooperative co-evolutionary bare-bones particle swarm optimization (CCBBPSO) was proposed by devising a function independent decomposition (FID) method to decompose the high-dimensional problem and then employing BBPSO to optimize the decomposed sub-problems. In [73], a two stages variable interaction reconstruction algorithm along with a learning model and a marginalized denoising model was first proposed to construct the overall variable interactions using prior knowledge to decompose a high-dimensional problem into sub-problems. Then, a cooperative hierarchical particle swarm optimization framework was designed to optimize the decomposed sub-problems.

Since the decomposed sub-problems are separately optimized and the optimization of interacting variables usually intertwines with each other, the decomposition strategy to divide a high-dimensional problem into sub-problems plays a crucial role in CCPSO. Therefore, recent research on CCEAs, including CCPSO, mainly focuses on devising effective decomposition strategies to divide a high-dimensional problem into sub-problems as accurately as possible. Along this line, many remarkable decomposition strategies [69,74,75,76,77] have been proposed. Among them, the most representative one is the differential grouping (DG) strategy [69], which adopts partial difference between function values by disturbing two variables to detect their interaction. After this decomposition method, many improved versions [74,75,76,78] have been devised to promote its detection accuracy and efficiency. For instance, in [79], a recursive differential grouping (RDG) was devised by employing a nonlinearity detection method to detect the interaction between decision variables. Specifically, it recursively examines the interaction between one decision variable and the remaining variables based on the idea of binary search. To further improve its detection efficiency, an efficient recursive differential grouping (ERDG) [75] was proposed, and to alleviate its sensitivity to parameters, an improved version, named RDG2, was developed [80]. In [74], an improved variant of DG, named DG2, was proposed to promote the efficiency and grouping accuracy of DG. In particular, it adopts a reliable threshold value by estimating the magnitude of round-off errors, and reuses the sample points generated for detecting interactions to save the consumption of fitness evaluations in the decomposition stage. In [76], Ma et al. proposed a merged differential grouping method based on subset–subset interaction and binary search. Specifically, in this algorithm, each variable is first identified as either a separable variable or a non-separable variable, and then all separable variables are put into the same subset, and the non-separable variables are divided into multiple subsets using a binary-tree-based iterative merging method. In [78], Liu et al. proposed a hybrid deep grouping (HDG) algorithm by considering the variable interaction and the essentialness of the variable. In [81], Zhang et al. even proposed a dynamic decomposition strategy by first designing a novel estimation method to evaluate the contribution of variables and then constructing the dynamic subcomponent based on the estimated contributions.

Although the above advanced decomposition strategies greatly improve the optimization performance of CCEAs including CCPSOs, they still encounter limitations especially when dealing with problems with overlapping interaction between variables and fully non-separable problems.

2.2.2. Holistic Large Scale PSOs

Different from CCPSO, holistic large-scale PSOs [29,30,31,32,56,82] still optimize all variables together like traditional PSOs. To effectively and efficiently search the exponentially increased solution space, the key to designing holistic large-scale PSOs is to maintain high search diversity, so that particles could search dispersedly in different directions to find promising areas fast. To this end, in the literature [29,30,31], researchers usually abandon the utilization of historical best positions, such as the personal best positions pbests, the global best position gbest, and the neighbor best positions nbests, to direct the update of particles because they usually remain unchanged in many generations, especially in the late stage of the evolutions. Instead, they generally directly employ the predominant particles in the current swarm to guide the update of poor particles.

Along this line, many remarkable novel large-scale PSO variants [29,30,31,54,55] have been developed by taking inspiration from intelligent behaviors of natural animals and human society. For instance, a competitive swarm optimizer (CSO) [29] was developed by imitating the competition mechanism in human society to tackle large-scale optimization. Specifically, this algorithm first randomly arranges particles into pairs and then lets each pair of particles compete with each other. After the competition, the winner is not updated and directly enters the next generation, while the loser is updated by learning from the winner and the mean position of the swarm. Inspired by the social learning strategy in animals, Cheng and Jin devised a social learning PSO (SLPSO) [56], which assigns each particle a learning probability and then lets each particle learn from the predominant ones in the swarm and the mean position of the swarm. To further improve the optimization performance of CSO, Yang et al. proposed a segment-based predominant learning swarm optimizer (SPLSO) [55] by dividing the whole dimension into several segments and using different predominant particles to guide different segments of the updated particle. Instead of the pairwise competition mechanism in CSO, Mohapatra et al. [83] designed a tri-competitive strategy, leading to a modified CSO (MCSO), to improve the learning effectiveness of particles. In [32], Kong et al. proposed an adaptive multi-swarm competition PSO (AMCPSO) by randomly dividing the whole swarm into several sub-swarms, and then using the competitive mechanism in each sub-swarm to update particles. In [31], a two-phase learning-based swarm optimizer (TPLSO) was proposed by taking inspiration from the cooperative learning behavior in human society. In this algorithm, a mass learning strategy and an elite learning mechanism were designed. In the former strategy, three particles are first randomly selected to form a study group and then the competitive mechanism is utilized to update the members of the study group. In the latter strategy, all particles are first sorted and the elite particles with better fitness values are picked out to learn from each other to exploit promising areas. In [84], a ranking-based biased learning swarm optimizer (RBLSO) was developed by maximizing the fitness difference between learners and exemplars. Specifically, in this algorithm, a ranking paired learning (RPL) strategy and a biased center learning (BCL) strategy were devised to update particles. In RPL, poor particles learn from predominant particles, while in BCL, each particle learns from the biased center, which is computed as the fitness weighted center of the whole swarm.

The above PSO variants have shown promising performance in solving large-scale optimization. However, in these variants, each particle is guided by only one predominant particle. To further promote the learning diversity and the learning effectiveness of particles, researchers have attempted to utilize two different predominant particles to guide the evolution of each particle. For instance, Yang et al. proposed a level-based learning swarm optimizer (LLSO) [54] by taking inspiration from pedagogy. Specifically, this algorithm first partitions particles in the swarm into several levels and then lets each particle in lower levels learn from two different predominant ones in higher levels. In [85], a particle swarm optimizer with multi-level population sampling and dynamic p-learning mechanisms was devised by first partitioning the particles in the swarm into multi-levels based on their fitness and then designing a dynamic p-learning mechanism to accelerate the learning of particles. In [30], an adaptive stochastic dominant learning swarm optimizer (SDLSO) was proposed to effectively tackle large-scale optimization. In this algorithm, each particle is compared with two randomly selected ones. Only when the particle is dominated by both of the two selected ones, it is updated by learning from them; otherwise, the particle is not updated and directly enters the next generation.

To improve the search efficiency of PSO in high-dimensional space, some researchers even proposed distributed parallel large-scale PSOs. For example, in [71], a distributed elite-guided learning swarm optimizer (DEGLSO) was devised by using a master-slave parallel model. Specifically, in this algorithm, a master process and multiple slave processes are maintained with each slave maintaining a small swarm to evolve in parallel to cooperatively search the high-dimensional space and the master responsible for communication among slaves. In [39], an adaptive granularity learning distributed particle swarm optimization (AGLDPSO) was proposed by also using the master-slave distributed model. In this algorithm, the entire swarm is first divided into multiple sub-swarms, and these sub-swarms are co-evolved in parallel by the slaves.

Though the above holistic large-scale PSOs have presented great optimization ability in solving large-scale optimization problems, they are still confronted with falling into local regions and premature convergence when tackling complicated high-dimensional problems with a lot of local optima. Therefore, the optimization performance of PSO in coping with high-dimensional problems still needs improving and thus the research on holistic large-scale PSO still deserves extensive attention. In this paper, inspired by the “Pareto Principle” [57], we propose an elite-directed particle swarm optimization with historical information (EDPSO) to tackle large-scale optimization.

3. Proposed Method

Elite individuals in one species usually preserve more valuable evolutionary information to guide the evolution of the species than others [86]. Taking inspiration from this, we devise an elite-directed PSO (EDPSO) with historical information to improve the learning effectiveness and the learning diversity of particles to search the large-scale solution space efficiently. Specifically, the main components of the proposed EDPSO are elucidated in detail as follows.

3.1. Elite-Directed Learning

Taking inspiration from the “Pareto Principle” [57] which is also known as the 80-20 rule that 80% of the consequences come from 20% of the causes, asserting an unequal relationship between inputs and outputs, we first divide the swarm with NP particles into two exclusive sets. One set is the elite set containing the top best 20% of particles, while another set is the non-elite set containing the remaining 80% of particles. To further treat different particles differently, we further divide the non-elite set equally into two layers. One layer consists of the best half of the non-elite set, while another layer contains the remaining half of the non-elite set. In this way, we actually separate particles in the swarm into three exclusive layers. The first layer, denoted as L₁, is the elite set, composed of the top best 20% of particles. The second layer, denoted as L₂, is the best half of the non-elite set, made up of 40% particles that are only inferior to those in L₁. The third layer, denoted as L₃, contains the remaining 40% of particles. On the whole, it is found that particles in the first layer (L₁) are the best, while those in the third layer (L₃) are the worst among the three layers.

Since particles in different layers preserve different strengths in exploring and exploiting the solution space, we treat them differently. Specifically, for particles in the third layer (L₃), since they are all dominated by those in the first two layers (L₁ and L₂), they are updated by learning from those predominant particles in L₁ and L₂. Likewise, for particles in the second layer (L₂), since they are all dominated by those in the first layer (L₁), they are updated by learning from those in L₁. As for particles in the first layer, since they are the best in the swarm, we do not update them and let them directly enter the next generation to preserve valuable evolutionary information and keep them from being destroyed, so that the evolution of the swarm can be guaranteed to converge to promising areas.

In particular, particles in the third layer (L₃) are updated as follows:

$${\mathit{v}}_{L_{3,i}}\leftarrow r_1{\mathit{v}}_{L_{3,i}}+r_2({\mathit{x}}_{L_{1,2,k1}}-{\mathit{x}}_{L_{3,i}})+r_3\varphi ({\mathit{x}}_{L_{1,2,k2}}-{\mathit{x}}_{L_{3,i}})$$

(4)

$${\mathit{x}}_{L_{3,i}}\leftarrow {\mathit{x}}_{L_{3,i}}+{\mathit{v}}_{L_{3,i}}$$

(5)

where ${\mathit{x}}_{L_{3,i}}=\left[x_{L_{3,i}}^1,x_{L_{3,i}}^2,x_{L_{3,i}}^3,\dots ,x_{L_{3,i}}^D\right]$ and ${\mathit{v}}_{L_{3,i}}=\left[v_{L_{3,i}}^1,v_{L_{3,i}}^2,v_{L_{3,i}}^3,\dots ,v_{L_{3,i}}^D\right]$ are the position and the velocity of the ith particle in the third layer (L₃), respectively. ${\mathit{x}}_{L_{1,2,k1}}$ and ${\mathit{x}}_{L_{1,2,k2}}$ are two different exemplars randomly selected from L₁∪L₂. $r_1$, $r_2$, and $r_3$ are three real random numbers uniformly generated within [0, 1]. ϕ is a control parameter within [0, 1], which is in charge of the influence of the second exemplar on the updated particle.

Likewise, particles in the second layer (L₂) are updated as follows:

$${\mathit{v}}_{L_{2,i}}\leftarrow r_1{\mathit{v}}_{L_{2,i}}+r_2({\mathit{x}}_{L_{1,k1}}-{\mathit{x}}_{L_{2,i}})+r_3\varphi ({\mathit{x}}_{L_{1,k2}}-{\mathit{x}}_{L_{2,i}})$$

(6)

$${\mathit{x}}_{L_{2,i}}\leftarrow {\mathit{x}}_{L_{2,i}}+{\mathit{v}}_{L_{2,i}}$$

(7)

where ${\mathit{x}}_{L_{2,i}}=\left[x_{L_{2,i}}^1,x_{L_{2,i}}^2,x_{L_{2,i}}^3,\dots ,x_{L_{2,i}}^D\right]$ and ${\mathit{v}}_{L_{2,i}}=\left[v_{L_{2,i}}^1,v_{L_{2,i}}^2,v_{L_{2,i}}^3,\dots ,v_{L_{2,i}}^D\right]$ are the position and the velocity of the ith particle in the second layer (L₂), respectively. ${\mathit{x}}_{L_{1,k1}}$ and ${\mathit{x}}_{L_{1,k2}}$ represent the two different exemplars randomly selected from the first layer (L₁).

In addition to the above updated formula, the following details deserve special attention:

(1)

For particles in L₂ and L₃, we randomly select two different predominant particles from higher layers to direct their update. In this way, for different particles, the two guiding exemplars are different. In addition, for the same particle, the two guiding exemplars are also different in different generations. Therefore, the learning diversity of particles is largely improved.

(2)

As for the two selected exemplars, we utilize the better one as the first guiding exemplar and the worse one as the second guiding exemplar. That is to say, in Equation (4), ${\mathit{x}}_{L_{1,2,k1}}$ is better than ${\mathit{x}}_{L_{1,2,k2}}$ and in Equation (6), ${\mathit{x}}_{L_{1,k1}}$ is better than ${\mathit{x}}_{L_{1,k2}}$ Such employment of the two exemplars results in the first guiding exemplar being mainly responsible for leading the update particle to approach promising areas, and thus is in charge of the convergence. At the same time, the second guiding exemplar is mainly responsible for diversity, so that the updated particle is prevented from approaching the first exemplar too greedily. In this way, a promising balance between fast convergence and high diversity is expectedly maintained at the particle level during the update of each particle.

(3)

With respect to Equations (5) and (7), once the elements in the position of the update particle are out of the range of the associated variables, they are set as the lower bounds of the associated variables if they are smaller than the associated lower bounds and are set as the upper bounds of the associated variables if they are larger than the associated upper bounds.

(4)

Taking deep observation, we find that particles in L₃ have a wider range to learn than those in L₂. This is because particles in L₃ learn from those in L₂∪L₁ consisting of the top best 60% of particles in the swarm, while particles in L₂ only learn from those in L₁ containing the top best 20% of particles. Therefore, we can see that during the evolution, particles in L₃ bias to exploring the solution space, while those in L₂ bias to exploiting the found promising areas. Hence, a promising balance between exploration and exploitation is expectedly maintained at the swarm level during the evolution of the whole swarm.

(5)

Since particles in L₁ are preserved to directly enter the next generation, they become better and better during the evolution, though the members in L₁ are likely changed. As a result, particles in L₁ are expected to converge to optimal areas.

3.2. Historical Information Utilization

During the evolution, some of the particles in L₁ in the last generation are replaced by some of the updated particles in L₂ and L₃, once they are better. Therefore, some particles in L₁ are obsolete and enter L₂ and L₃ to update in the next generation. However, these obsolete solutions may also contain useful evolutionary information. To make full use of the historical information, we maintain an additional archive A of size NP/2 to store the obsolete elite individuals.

Specifically, before the update of particles in each generation, when the three layers are formed, we first compute L_1,g∩L_1,g−1 to obtain the elite particles that survive to directly enter the next generation. Then, we calculate L_1,g–1 − (L_1,g∩L_1,g–1) to obtain the obsolete elites. These obsolete elites are stored in the archive A.

As for the update of the archive A, in the beginning, A is set to be empty. Then, the obsolete elites (L_1,g–1 − (L_1,g∩L_1,g–1)) are inserted into A from the worst to the best. Once the archive A is full, namely its size exceeds NP/2, we adopt the “first-in-first-out” strategy to remove individuals in A until its size reaches NP/2.

To make use of the obsolete elites in A reasonably, we only introduce them into the learning of particles in L₃, so that the convergence of the swarm is not destroyed. Specifically, to guarantee that particles in L₃ learn from better ones, we first seek those individuals in A that are better than the best one in L₃ to form a candidate set A’. Then, A’ is utilized to update particles in L₃ along with the first two layers. That is to say, in Equation (4), the two guiding exemplars are randomly selected from L₁∪L₂∪A’ instead of only from L₁∪L₂.

With the above modification, the number of candidate exemplars for particles in L₃ is enlarged, and thus the learning diversity of particles in L₃ is further promoted to fully explore the high-dimensional solution space. It should be mentioned that on the one hand, particles in L₃ are guaranteed to learn from better ones; on the other hand, the learning of particles in L₂, which are bias to exploiting promising areas, remains unchanged. Therefore, the convergence of the swarm is not destroyed.

Experiments conducted in Section 4.3 will demonstrate the usefulness of the additional archive A in helping the proposed EDPSO to achieve a promising performance.

3.3. Overall Procedure of EDPSO

Integrating the above components, the complete EDPSO is obtained with its overall flowchart exhibited in Figure 1 and the overall procedure presented in Algorithm 1. Specifically, in the beginning, NP particles are randomly initialized and evaluated as shown in Line 1. After that, the algorithm jumps into the main iteration (Lines 3–25). During the iteration, the whole swarm is sorted in an ascending order of fitness (Line 4), and then the swarm is partitioned into three layers (Line 5) along with the update of the archive A (Line 6). Subsequently, particles in L₃ are updated (Lines 7–16), following which is the update of particles in L₂ (Lines 17–25). The main iteration proceeds until the maximum number of fitness evaluations is exhausted. Then, at the end of the algorithm, the global best position gbest and its fitness f(gbest) are obtained from L₁ as the output.

From Algorithm 1, we can see that EDPSO needs O(NP × D) to store the positions of all particles and O(NP × D) to store the velocity of all particles. In addition, it also needs O(NP × D) to store the archive, and O(NP) to store the sorted index of particles. Comprehensively, EDPSO needs O(NP × D) space during the evolution.

Concerning the time complexity, in each generation, it takes O(NPlogNP) time to sort the swarm (Lines 4) and O(NP) to divide the swarm into three layers (Lines 5–6). Then, it takes O(NP × D) to update the archive (Lines 6), and to update particles in L₂ and L₃ (Lines 7–25) O(NP × D) time complexity is also needed. To sum up, the whole complexity in one generation is O(NP × D), which is the same as the classical PSO.

Algorithm 1: The pseudocode of EDPSO
Input: swarm size NP, maximum fitness evaluations FESmax and control parameter ∅
1:	Initialize NP particles randomly and caculate their fitness;
2:	FEs = NP and set the archive empty A = ∅
3:	While (FEs < FESmax) do
4:	Sort particles in ascending order according to their fitness;
5:	Partition the swarm into three layers: the elite layer L1 containing the top best 20% of particles, the second layer L2 containing the half best of the non-elite particles (40% particles), and the third layer L3 consisting of another half of the non-elite particles (40% particles);
6:	Update the archive A;
7:	For each particle in L3 do
8:	Find those individuals in A that are better than the best particle in L3 to form a set A’;
9:	Select two different exemplars xk1 and xk2 from L1∪L2∪A’;
10:	If f(xk1) > f(xk2) then
11:	Swap(xk1, xk2);
12:	End If
13:	Update this particle according to Equations (4) and (5);
14:	If the elements in xi are smaller than the lower bounds of the associated variables, they are set as the asociated lower bounds; if they are larger than the upper bounds of the associated variables, they are set as the associated upper bounds.
15:	Evaluate its fitness and FEs++;
16:	End For
17:	For each particle in L2 do
18:	Select two different exemplars xk1 and xk2 from L1;
19:	If f(xk1) > f(xk2) then
20:	Swap(xk1, xk2);
21:	End If
22:	Update this particle according to Equations (6) and (7);
23:	If the elements in xi are smaller than the lower bounds of the associated variables, they are set as the asociated lower bounds; if they are larger than the upper bounds of the associated variables, they are set as the associated upper bounds.
24:	Evaluate its fitness and FEs++;
25:	End For
26:	End While
27:	Obtain the global best position gbest and its fitness f(gbest);
Output: f(gbest) and gbest

4. Experiments

In this section, we conduct extensive comparison experiments to validate the effectiveness of the proposed EDPSO on the commonly used CEC’2010 [58] and CEC’2013 [59] large-scale benchmark problem sets by comparing EDPSO with several state-of-the-art large-scale optimizers. The CEC’2010 set contains twenty 1000-D optimization problems, while the CEC’2013 set consists of fifteen 1000-D problems. In particular, the CEC’2013 set is an extension of the CEC’2010 set by introducing more complicated features, such as overlapping interacting variables and unbalanced contribution of variables. Therefore, the CEC’2013 benchmark problems are more difficult to optimize. The main characteristics of the CEC’2010 and the CEC’2013 sets are briefly summarized in

Table 1. The main characteristics of the CEC’2010 functions.

F	Dimension	Separability	Optima	Modality
$F_1$	1000	fully separable	0	unimodal
$F_2$	1000	fully separable	0	multimodal
$F_3$	1000	fully separable	0	multimodal
$F_4$	1000	partially separable	0	unimodal
$F_5$	1000	partially separable	0	multimodal
$F_6$	1000	partially separable	0	multimodal
$F_7$	1000	partially separable	0	unimodal
	1000	partially separable	0	multimodal
$F_9$	1000	partially separable	0	unimodal
	1000	partially separable	0	multimodal
$F_{11}$	1000	partially separable	0	multimodal
$F_{12}$	1000	partially separable	0	unimodal
$F_{13}$	1000	partially separable	0	multimodal
$F_{14}$	1000	partially separable	0	unimodal
	1000	partially separable	0	multimodal
$F_{16}$	1000	partially separable	0	multimodal
$F_{17}$	1000	partially separable	0	unimodal
$F_{18}$	1000	partially separable	0	multimodal
$F_{19}$	1000	fully non-separable	0	unimodal
$F_{20}$	1000	fully non-separable	0	multimodal

and

Table 2. The main characteristics of the CEC’2013 functions.

F	Dimension	Separability	Optima	Modality
$F_1$	1000	fully separable	0	unimodal
$F_2$	1000	fully separable	0	multimodal
$F_3$	1000	fully separable	0	multimodal
$F_4$	1000	partially separable	0	unimodal
$F_5$	1000	partially separable	0	multimodal
$F_6$	1000	partially separable	0	multimodal
$F_7$	1000	partially separable	0	multimodal
$F_8$	1000	partially separable	0	unimodal
$F_9$	1000	partially separable	0	multimodal
$F_{10}$	1000	partially separable	0	multimodal
$F_{11}$	1000	partially separable	0	unimodal
$F_{12}$	1000	overlapping	0	multimodal
$F_{13}$	905	overlapping	0	unimodal
$F_{14}$	905	overlapping	0	unimodal
$F_{15}$	1000	fully non-separable	0	unimodal

, respectively. For more details, please refer to [58,59].

In this section, we first investigate the parameter settings of the swarm size NP and the control parameter ϕ for EDPSO in Section 4.1. Then, in Section 4.2, the proposed EDPSO is extensively compared with several state-of-the-art large-scale algorithms on the two benchmark sets. Lastly, in Section 4.3, the effectiveness of the additional archive in EDPSO is verified by conducting experiments on the CEC’2010 benchmark set.

In the experiments, for fair comparisons, without being otherwise stated, the maximum number of fitness evaluations is set as 3000 × D (where D is the dimension size) for all algorithms. For each algorithm on each optimization problem, we execute it independently for 30 runs, and then utilize the median, the mean, and the standard deviation (Std) values over the 30 independent runs to evaluate its performance, so that fair and comprehensive comparisons can be made.

In addition, to tell whether there is significant difference between the proposed EDPSO and the compared methods, we run the Wilcoxon rank-sum test at the significance level of “α = 0.05” to compare the performance of EDPSO with that of each compared algorithm on each problem. Furthermore, to compare the overall performance of different algorithms on one whole benchmark set, we also conduct the Friedman test at the significance level of “α = 0.05” on each benchmark set.

4.1. Parameter Settings

In the proposed EDPSO, two parameters, namely the swarm size NP and the control parameter ϕ, are needed to fine-tune. Therefore, to find suitable settings for these two parameters for EDPSO, we conduct experiments on the CEC’2010 set by varying NP from 500 to 1000, and ϕ from 0.1 to 0.9 for EDPSO. The comparison results among these settings are shown in Table 1. In particular, in this table, the best results among different settings of ϕ under the same setting of NP are highlighted in bold. The average rank of each configuration of ϕ under the same setting of NP obtained by the Friedman test is also presented in the last row of each part in Table 1.

From

Table 3. Comparison results of EDPSO with different settings of NP and ϕ on the 1000-D CEC’2010 problems with respect to the average fitness of the global best solutions found in 30 independent runs. In each part of this table with the same settings of NP, the best results obtained by EDPSO with the optimal ϕ are highlighted in bold.

F	NP = 500									F	NP = 600
	ϕ = 0.1	ϕ = 0.2	ϕ = 0.3	ϕ = 0.4	ϕ = 0.5	ϕ = 0.6	ϕ = 0.7	ϕ = 0.8	ϕ = 0.9		ϕ = 0.1	ϕ = 0.2	ϕ = 0.3	ϕ = 0.4	ϕ = 0.5	ϕ = 0.6	ϕ = 0.7	ϕ = 0.8	ϕ = 0.9
F1	1.33 × 10−21	1.57 × 10−22	4.66 × 10−23	1.89 × 10−23	1.86 × 10−22	9.58 × 10−12	2.58 × 102	2.83 × 107	1.31 × 109	F1	4.60 × 10−21	1.51 × 10−22	4.52 × 10−23	2.77 × 10−23	2.32 × 10−21	3.17 × 10−7	2.12 × 104	8.43 × 107	1.97 × 109
F2	1.23 × 103	1.54 × 103	1.72 × 103	1.30 × 103	9.12 × 102	1.04 × 104	1.08 × 104	1.12 × 104	1.19 × 104	F2	1.02 × 103	1.33 × 103	1.32 × 103	1.10 × 103	2.66 × 103	1.04 × 104	1.07 × 104	1.14 × 104	1.19 × 104
F3	3.12 × 10−14	2.89 × 10−14	3.12 × 10−14	2.18 × 10−14	2.65 × 10−14	2.65 × 10−9	2.56 × 10−2	7.98 × 100	1.55 × 101	F3	4.43 × 10−14	2.65 × 10−14	3.01 × 10−14	2.18 × 10−14	3.60 × 10−14	6.21 × 10−7	4.36 × 10−1	1.05 × 101	1.63 × 101
F4	5.36 × 1011	5.54 × 1011	3.89 × 1011	3.72 × 1011	2.43 × 1011	2.54 × 1011	4.07 × 1011	3.12 × 1013	8.64 × 1013	F4	6.17 × 1011	5.52 × 1011	5.08 × 1011	4.14 × 1011	2.90 × 1011	2.28 × 1011	1.29 × 1012	4.35 × 1013	1.18 × 1014
F5	2.77 × 108	2.85 × 108	2.84 × 108	2.94 × 108	2.87 × 108	2.85 × 108	3.15 × 108	3.22 × 108	3.31 × 108	F5	2.74 × 108	2.86 × 108	2.78 × 108	2.85 × 108	2.88 × 108	2.94 × 108	2.98 × 108	2.98 × 108	3.33 × 108
F6	4.00 × 10−9	6.71 × 100	1.55 × 101	4.00 × 10−9	4.00 × 10−9	4.03 × 10−8	5.59 × 10−2	9.28 × 100	1.65 × 101	F6	4.01 × 10−9	4.00 × 10−9	4.00 × 10−9	4.00 × 10−9	4.00 × 10−9	2.81 × 10−6	7.87 × 10−1	1.18 × 101	1.74 × 101
F7	2.85 × 101	3.19 × 101	4.34 × 101	4.86 × 100	1.94 × 100	6.72 × 102	3.45 × 104	2.42 × 107	2.12 × 109	F7	1.65 × 102	8.34 × 101	7.31 × 101	7.50 × 100	1.60 × 101	2.85 × 103	1.57 × 105	8.17 × 107	2.45 × 109
F8	3.01 × 107	2.58 × 107	2.40 × 107	2.04 × 107	2.16 × 107	3.21 × 107	4.14 × 107	4.52 × 107	4.66 × 107	F8	3.31 × 107	2.88 × 107	2.70 × 107	2.39 × 107	2.58 × 107	3.46 × 107	4.25 × 107	4.56 × 107	4.65 × 107
F9	4.59 × 107	5.19 × 107	5.67 × 107	3.95 × 107	3.96 × 107	8.27 × 107	1.10 × 109	1.47 × 1010	3.01 × 1010	F9	4.87 × 107	5.06 × 107	5.41 × 107	3.78 × 107	4.15 × 107	1.10 × 108	3.69 × 109	1.81 × 1010	3.50 × 1010
F10	1.23 × 103	1.61 × 103	1.63 × 103	1.30 × 103	9.83 × 103	1.04 × 104	1.08 × 104	1.13 × 104	1.17 × 104	F10	9.20 × 103	1.29 × 103	1.39 × 103	1.53 × 103	1.02 × 104	1.05 × 104	1.08 × 104	1.14 × 104	1.19 × 104
F11	1.99 × 10−13	1.77 × 101	2.03 × 101	2.03 × 101	1.61 × 10−13	3.54 × 10−7	6.75 × 10−1	4.56 × 101	1.35 × 102	F11	3.94 × 10−13	5.70 × 10−1	1.65 × 101	1.54 × 10−13	2.82 × 10−13	1.41 × 10−5	4.62 × 100	6.64 × 101	1.49 × 102
F12	2.86 × 104	1.98 × 104	2.30 × 104	1.07 × 104	3.08 × 104	2.05 × 106	4.44 × 106	5.47 × 106	6.72 × 106	F12	4.78 × 104	2.47 × 104	2.66 × 104	1.73 × 104	7.31 × 104	2.99 × 106	4.55 × 106	5.63 × 106	6.89 × 106
F13	8.24 × 102	7.31 × 102	7.46 × 102	5.73 × 102	5.53 × 102	4.94 × 102	8.23 × 103	2.25 × 107	5.95 × 109	F13	7.83 × 102	6.06 × 102	6.05 × 102	5.50 × 102	5.17 × 102	5.57 × 102	7.41 × 103	1.32 × 108	1.09 × 1010
F14	1.31 × 108	1.40 × 108	1.57 × 108	1.04 × 108	1.12 × 108	3.04 × 108	5.87 × 109	3.11 × 1010	5.53 × 1010	F14	1.56 × 108	1.45 × 108	1.56 × 108	1.17 × 108	1.24 × 108	5.06 × 108	1.20 × 1010	3.80 × 1010	6.13 × 1010
F15	1.06 × 104	1.08 × 104	1.07 × 104	1.06 × 104	1.05 × 104	1.06 × 104	1.08 × 104	1.13 × 104	1.18 × 104	F15	1.05 × 104	1.05 × 104	1.06 × 104	1.06 × 104	1.04 × 104	1.05 × 104	1.08 × 104	1.13 × 104	1.19 × 104
F16	1.48 × 100	5.44 × 100	6.67 × 100	1.77 × 100	2.93 × 10−1	8.78 × 10−8	8.68 × 10−1	1.66 × 102	3.21 × 102	F16	2.93 × 10−1	2.65 × 100	1.51 × 100	1.75 × 10−13	3.06 × 10−13	1.82 × 10−5	1.11 × 101	2.18 × 102	3.36 × 102
F17	2.33 × 105	1.29 × 105	1.31 × 105	8.96 × 104	5.05 × 105	4.98 × 106	8.44 × 106	1.19 × 107	1.48 × 107	F17	4.51 × 105	1.59 × 105	1.46 × 105	1.39 × 105	2.11 × 106	5.81 × 106	9.40 × 106	1.24 × 107	1.50 × 107
F18	2.72 × 103	3.54 × 103	3.05 × 103	2.04 × 103	1.55 × 103	2.76 × 103	1.00 × 105	6.35 × 109	8.50 × 1010	F18	2.01 × 103	2.17 × 103	3.00 × 103	2.45 × 103	1.58 × 103	2.12 × 103	3.91 × 106	1.34 × 1010	9.69 × 1010
F19	9.54 × 106	6.22 × 106	6.10 × 106	7.73 × 106	1.09 × 107	1.42 × 107	1.70 × 107	2.27 × 107	2.53 × 107	F19	1.03 × 107	7.37 × 106	6.84 × 106	8.92 × 106	1.14 × 107	1.51 × 107	1.95 × 107	2.25 × 107	2.64 × 107
F20	1.65 × 103	2.05 × 103	2.28 × 103	1.53 × 103	1.27 × 103	1.25 × 103	6.44 × 104	6.85 × 109	8.27 × 1010	F20	1.75 × 103	1.83 × 103	1.94 × 103	1.31 × 103	1.18 × 103	1.05 × 103	3.89 × 106	1.51 × 1010	1.08 × 1011
Rank	3.53	4.15	4.43	2.45	2.55	4.50	6.45	7.95	9.00	Rank	3.90	3.63	3.68	2.05	3.05	4.75	6.95	8.00	9.00
F	NP = 700									F	NP = 800
	ϕ = 0.1	ϕ = 0.2	ϕ = 0.3	ϕ = 0.4	ϕ = 0.5	ϕ = 0.6	ϕ = 0.7	ϕ = 0.8	ϕ = 0.9		ϕ = 0.1	ϕ = 0.2	ϕ = 0.3	ϕ = 0.4	ϕ = 0.5	ϕ = 0.6	ϕ = 0.7	ϕ = 0.8	ϕ = 0.9
F1	8.23 × 10−18	2.92 × 10−22	6.67 × 10−23	7.69 × 10−23	3.84 × 10−18	4.62 × 10−4	1.92 × 105	1.80 × 108	2.84 × 109	F1	3.60 × 10−14	6.27 × 10−22	1.10 × 10−22	4.98 × 10−22	3.99 × 10−14	7.53 × 10−2	9.19 × 105	2.76 × 108	3.52 × 109
F2	8.38 × 102	1.18 × 103	1.22 × 103	9.67 × 102	9.99 × 103	1.03 × 104	1.08 × 104	1.14 × 104	1.20 × 104	F2	1.27 × 103	1.02 × 103	1.09 × 103	8.38 × 102	1.01 × 104	1.05 × 104	1.10 × 104	1.14 × 104	1.20 × 104
F3	2.42 × 10−12	2.77 × 10−14	2.65 × 10−14	2.29 × 10−14	1.60 × 10−12	2.38 × 10−5	2.07 × 100	1.23 × 101	1.68 × 101	F3	1.76 × 10−10	2.89 × 10−14	2.89 × 10−14	2.65 × 10−14	1.69 × 10−10	3.43 × 10−4	3.48 × 100	1.32 × 101	1.72 × 101
F4	6.68 × 1011	8.31 × 1011	4.07 × 1011	4.72 × 1011	2.88 × 1011	3.39 × 1011	1.59 × 1012	3.88 × 1013	1.38 × 1014	F4	5.44 × 1011	7.74 × 1011	6.56 × 1011	5.49 × 1011	4.23 × 1011	3.81 × 1011	2.15 × 1012	4.92 × 1013	1.09 × 1014
F5	2.81 × 108	2.81 × 108	2.84 × 108	2.86 × 108	2.78 × 108	2.95 × 108	3.00 × 108	2.98 × 108	3.15 × 108	F5	2.76 × 108	2.84 × 108	2.80 × 108	2.89 × 108	2.89 × 108	2.99 × 108	2.92 × 108	3.19 × 108	3.25 × 108
F6	4.48 × 10−9	4.00 × 10−9	4.00 × 10−9	4.00 × 10−9	4.25 × 10−9	9.58 × 10−5	2.65 × 100	1.34 × 101	1.80 × 101	F6	1.27 × 10−8	4.00 × 10−9	4.00 × 10−9	4.00 × 10−9	1.27 × 10−8	1.16 × 10−3	4.15 × 100	1.42 × 101	1.82 × 101
F7	4.12 × 102	1.76 × 102	2.47 × 102	4.95 × 101	1.06 × 102	1.05 × 104	5.97 × 105	2.68 × 108	3.69 × 109	F7	1.55 × 103	6.37 × 102	1.08 × 103	1.63 × 102	3.23 × 102	2.52 × 104	1.04 × 106	5.36 × 108	4.64 × 109
F8	3.50 × 107	3.12 × 107	2.95 × 107	2.69 × 107	2.90 × 107	3.62 × 107	4.33 × 107	4.54 × 107	4.66 × 107	F8	3.63 × 107	3.30 × 107	3.14 × 107	2.92 × 107	3.09 × 107	3.76 × 107	4.35 × 107	4.59 × 107	4.67 × 107
F9	6.27 × 107	5.23 × 107	5.25 × 107	4.09 × 107	4.73 × 107	1.60 × 108	6.71 × 109	2.25 × 1010	3.59 × 1010	F9	6.85 × 107	5.83 × 107	5.27 × 107	4.63 × 107	5.40 × 107	2.63 × 108	9.57 × 109	2.40 × 1010	3.97 × 1010
F10	9.85 × 103	1.16 × 103	1.26 × 103	3.79 × 103	1.03 × 104	1.06 × 104	1.08 × 104	1.14 × 104	1.19 × 104	F10	1.01 × 104	1.02 × 103	1.02 × 103	9.96 × 103	1.03 × 104	1.05 × 104	1.09 × 104	1.15 × 104	1.21 × 104
F11	6.45 × 10−11	1.49 × 10−13	1.45 × 10−13	1.41 × 10−13	5.30 × 10−11	4.23 × 10−4	9.31 × 100	7.36 × 101	1.57 × 102	F11	3.81 × 10−9	1.42 × 10−13	1.35 × 10−13	1.41 × 10−13	4.52 × 10−9	5.26 × 10−3	1.56 × 101	8.86 × 101	1.65 × 102
F12	8.34 × 104	3.22 × 104	3.19 × 104	2.56 × 104	1.99 × 105	3.38 × 106	4.97 × 106	6.12 × 106	7.05 × 106	F12	1.34 × 105	4.03 × 104	3.35 × 104	3.69 × 104	6.23 × 105	3.43 × 106	5.13 × 106	6.09 × 106	7.35 × 106
F13	6.12 × 102	7.30 × 102	7.43 × 102	8.62 × 102	5.05 × 102	4.68 × 102	5.39 × 104	4.47 × 108	1.33 × 1010	F13	4.94 × 102	6.67 × 102	6.91 × 102	7.08 × 102	4.91 × 102	4.72 × 102	2.28 × 105	8.72 × 108	1.88 × 1010
F14	1.80 × 108	1.52 × 108	1.48 × 108	1.14 × 108	1.46 × 108	9.38 × 108	1.76 × 1010	4.20 × 1010	6.65 × 1010	F14	2.13 × 108	1.54 × 108	1.55 × 108	1.21 × 108	1.85 × 108	1.78 × 109	2.40 × 1010	4.72 × 1010	6.86 × 1010
F15	1.05 × 104	1.05 × 104	1.05 × 104	1.04 × 104	1.03 × 104	1.06 × 104	1.10 × 104	1.14 × 104	1.19 × 104	F15	1.04 × 104	1.05 × 104	1.05 × 104	1.04 × 104	1.04 × 104	1.06 × 104	1.10 × 104	1.14 × 104	1.21 × 104
F16	7.97 × 10−11	1.94 × 10−13	5.86 × 10−1	2.93 × 10−1	3.92 × 10−11	6.55 × 10−4	5.14 × 101	2.49 × 102	3.40 × 102	F16	6.70 × 10−9	2.03 × 10−13	3.42 × 10−1	2.07 × 10−13	4.18 × 10−9	8.11 × 10−3	7.54 × 101	2.66 × 102	3.48 × 102
F17	1.02 × 106	2.06 × 105	1.75 × 105	2.40 × 105	2.77 × 106	6.90 × 106	9.55 × 106	1.21 × 107	1.61 × 107	F17	2.33 × 106	2.73 × 105	2.10 × 105	4.21 × 105	3.90 × 106	7.36 × 106	9.71 × 106	1.32 × 107	1.55 × 107
F18	2.14 × 103	2.57 × 103	2.31 × 103	1.77 × 103	2.05 × 103	2.62 × 103	4.57 × 107	2.28 × 1010	1.23 × 1011	F18	1.97 × 103	1.90 × 103	2.13 × 103	1.59 × 103	1.20 × 103	1.75 × 103	2.35 × 108	3.15 × 1010	1.43 × 1011
F19	1.16 × 107	8.09 × 106	7.83 × 106	9.57 × 106	1.17 × 107	1.60 × 107	2.00 × 107	2.12 × 107	2.69 × 107	F19	1.22 × 107	9.60 × 106	8.07 × 106	9.79 × 106	1.25 × 107	1.70 × 107	1.97 × 107	2.33 × 107	2.90 × 107
F20	1.14 × 103	1.69 × 103	1.83 × 103	1.34 × 103	1.15 × 103	1.01 × 103	4.97 × 107	2.36 × 1010	1.23 × 1011	F20	1.15 × 103	1.40 × 103	1.67 × 103	1.10 × 103	1.03 × 103	1.35 × 103	2.81 × 108	3.22 × 1010	1.42 × 1011
Rank	4.00	3.13	3.15	2.48	3.05	5.20	7.05	7.95	9.00	Rank	3.95	3.18	2.93	2.20	3.55	5.25	6.95	8.00	9.00
F	NP = 900									F	NP = 1000
	ϕ = 0.1	ϕ = 0.2	ϕ = 0.3	ϕ = 0.4	ϕ = 0.5	ϕ = 0.6	ϕ = 0.7	ϕ = 0.8	ϕ = 0.9		ϕ = 0.1	ϕ = 0.2	ϕ = 0.3	ϕ = 0.4	ϕ = 0.5	ϕ = 0.6	ϕ = 0.7	ϕ = 0.8	ϕ = 0.9
F1	2.88 × 10−11	2.30 × 10−21	3.55 × 10−22	3.82 × 10−21	4.57 × 10−11	2.99 × 100	2.72 × 106	4.24 × 108	4.36 × 109	F1	5.39 × 10−9	7.77 × 10−20	1.84 × 10−21	1.08 × 10−18	1.13 × 10−8	4.65 × 101	6.59 × 106	5.75 × 108	4.98 × 109
F2	8.01 × 103	9.43 × 102	9.92 × 102	7.73 × 102	1.02 × 104	1.06 × 104	1.09 × 104	1.16 × 104	1.21 × 104	F2	9.60 × 103	8.89 × 102	9.57 × 102	7.69 × 103	1.02 × 104	1.05 × 104	1.10 × 104	1.15 × 104	1.22 × 104
F3	5.03 × 10−9	3.36 × 10−14	2.77 × 10−14	3.95 × 10−14	5.98 × 10−9	2.51 × 10−3	4.55 × 100	1.40 × 101	1.73 × 101	F3	7.68 × 10−8	1.80 × 10−13	3.01 × 10−14	8.03 × 10−13	1.09 × 10−7	1.12 × 10−2	5.60 × 100	1.45 × 101	1.75 × 101
F4	7.99 × 1011	7.13 × 1011	6.84 × 1011	4.70 × 1011	4.30 × 1011	4.15 × 1011	4.18 × 1012	6.47 × 1013	1.52 × 1014	F4	9.19 × 1011	9.07 × 1011	9.28 × 1011	6.61 × 1011	4.91 × 1011	5.14 × 1011	1.22 × 1013	7.49 × 1013	1.71 × 1014
F5	2.81 × 108	2.87 × 108	2.81 × 108	2.88 × 108	2.90 × 108	2.96 × 108	3.04 × 108	3.21 × 108	3.05 × 108	F5	2.91 × 108	2.87 × 108	2.74 × 108	2.91 × 108	2.77 × 108	2.90 × 108	2.99 × 108	3.24 × 108	3.28 × 108
F6	8.16 × 10−8	4.00 × 10−9	4.00 × 10−9	4.00 × 10−9	8.88 × 10−8	7.56 × 10−3	5.51 × 100	1.51 × 101	1.86 × 101	F6	7.90 × 10−7	4.11 × 10−9	4.00 × 10−9	4.20 × 10−9	9.85 × 10−7	3.28 × 10−2	6.85 × 100	1.55 × 101	1.89 × 101
F7	4.34 × 103	1.64 × 103	1.48 × 103	6.09 × 102	1.88 × 103	4.68 × 104	3.75 × 106	6.39 × 108	5.22 × 109	F7	1.07 × 104	6.59 × 103	5.32 × 103	1.33 × 103	3.16 × 103	7.82 × 104	8.34 × 106	1.06 × 109	5.69 × 109
F8	3.75 × 107	3.44 × 107	3.30 × 107	3.10 × 107	3.26 × 107	3.88 × 107	4.40 × 107	4.62 × 107	4.67 × 107	F8	3.84 × 107	3.56 × 107	3.42 × 107	3.24 × 107	3.40 × 107	3.94 × 107	4.42 × 107	4.62 × 107	4.67 × 107
F9	6.99 × 107	5.67 × 107	5.77 × 107	4.59 × 107	6.41 × 107	5.03 × 108	1.11 × 1010	2.51 × 1010	4.35 × 1010	F9	8.37 × 107	6.27 × 107	6.76 × 107	5.13 × 107	7.33 × 107	9.70 × 108	1.28 × 1010	3.10 × 1010	4.55 × 1010
F10	1.01 × 104	3.96 × 103	1.69 × 103	1.02 × 104	1.03 × 104	1.06 × 104	1.10 × 104	1.15 × 104	1.22 × 104	F10	1.02 × 104	9.41 × 103	9.35 × 103	1.02 × 104	1.03 × 104	1.06 × 104	1.09 × 104	1.16 × 104	1.22 × 104
F11	1.11 × 10−7	2.78 × 10−13	1.35 × 10−13	6.80 × 10−13	1.44 × 10−7	2.98 × 10−2	2.22 × 101	1.03 × 102	1.68 × 102	F11	1.41 × 10−6	7.70 × 10−12	2.52 × 10−13	2.49 × 10−11	2.15 × 10−6	1.19 × 10−1	2.59 × 101	1.09 × 102	1.73 × 102
F12	2.26 × 105	5.21 × 104	4.08 × 104	5.52 × 104	1.33 × 106	3.62 × 106	4.90 × 106	6.21 × 106	7.22 × 106	F12	3.83 × 105	6.13 × 104	4.93 × 104	7.39 × 104	1.72 × 106	3.75 × 106	5.27 × 106	6.09 × 106	7.39 × 106
F13	5.92 × 102	7.06 × 102	5.32 × 102	6.44 × 102	4.75 × 102	4.77 × 102	7.53 × 105	1.59 × 109	2.30 × 1010	F13	5.02 × 102	5.78 × 102	5.37 × 102	5.72 × 102	5.23 × 102	5.10 × 102	2.54 × 106	2.45 × 109	2.90 × 1010
F14	2.41 × 108	1.69 × 108	1.68 × 108	1.35 × 108	2.46 × 108	3.45 × 109	2.75 × 1010	5.28 × 1010	7.32 × 1010	F14	2.88 × 108	1.80 × 108	1.77 × 108	1.50 × 108	3.07 × 108	5.52 × 109	3.11 × 1010	5.41 × 1010	7.56 × 1010
F15	1.04 × 104	1.04 × 104	1.05 × 104	1.04 × 104	1.03 × 104	1.06 × 104	1.10 × 104	1.15 × 104	1.21 × 104	F15	1.04 × 104	1.03 × 104	1.03 × 104	1.03 × 104	1.03 × 104	1.07 × 104	1.10 × 104	1.16 × 104	1.23 × 104
F16	1.16 × 10−7	2.77 × 10−13	1.97 × 10−13	4.64 × 10−13	1.43 × 10−7	6.00 × 10−2	9.28 × 101	2.82 × 102	3.51 × 102	F16	1.75 × 10−6	4.05 × 10−12	2.67 × 10−13	1.78 × 10−11	2.51 × 10−6	2.39 × 10−1	1.19 × 102	2.95 × 102	3.58 × 102
F17	2.70 × 106	3.51 × 105	2.75 × 105	8.26 × 105	4.26 × 106	7.45 × 106	1.04 × 107	1.33 × 107	1.61 × 107	F17	3.49 × 106	5.20 × 105	3.59 × 105	1.52 × 106	4.78 × 106	7.70 × 106	1.02 × 107	1.29 × 107	1.60 × 107
F18	1.28 × 103	1.49 × 103	2.04 × 103	1.76 × 103	1.22 × 103	5.19 × 103	6.05 × 108	3.93 × 1010	1.71 × 1011	F18	1.39 × 103	1.82 × 103	1.80 × 103	1.51 × 103	1.57 × 103	7.40 × 103	1.51 × 109	4.85 × 1010	1.73 × 1011
F19	1.21 × 107	9.49 × 106	9.12 × 106	1.06 × 107	1.34 × 107	1.66 × 107	2.09 × 107	2.27 × 107	2.63 × 107	F19	1.21 × 107	9.70 × 106	9.89 × 106	1.08 × 107	1.32 × 107	1.60 × 107	1.97 × 107	2.35 × 107	2.77 × 107
F20	1.13 × 103	1.58 × 103	1.73 × 103	1.09 × 103	1.03 × 103	1.28 × 103	6.96 × 108	4.21 × 1010	1.67 × 1011	F20	1.09 × 103	1.15 × 103	1.14 × 103	9.99 × 102	1.10 × 103	4.70 × 103	1.73 × 109	5.17 × 1010	1.84 × 1011
Rank	3.90	2.80	2.30	2.75	3.80	5.45	7.00	8.05	8.95	Rank	3.95	2.90	2.25	2.55	3.85	5.50	7.00	8.00	9.00

, we can obtain the following findings. (1) For different settings of NP, we find that when NP ranges from 500 to 800, the most suitable ϕ is 0.4; when it ranges from 900 to 1000, the most suitable ϕ is 0.3. This indicates that the setting of ϕ is not so closely related to NP. (2) Furthermore, we find that neither a too small ϕ nor a too large ϕ is suitable for EDPSO. This is because a too small ϕ decreases the influence of the second exemplar on the updated particle, leading to the updated particle greedily approaching the promising area where the first exemplar is located. In this case, once the first exemplar falls into local areas, the updated particle likely falls into local basins as well. Therefore, the algorithm likely encounters premature convergence. On the contrary, a too large ϕ increases the influence of the second exemplar, leading to the updated particle being dragged too far away from promising areas. In this situation, the convergence of the swarm slows down. (3) It is observed that a relatively large NP is preferred for EDPSO to achieve good performance. This is because a small NP cannot afford enough diversity for the algorithm to explore the solution space. However, a too large NP (such as NP = 900 or 1000) is not beneficial for EDPSO to achieve good performance because too high diversity is afforded, leading to that the convergence of the swarm slows down.

Based on the above observation, NP = 600 and ϕ = 0.4 are the recommended settings for the two parameters in EDPSO when solving 1000-D optimization problems.

4.2. Comparisons with State-of-the-Art Large-Scale Optimizers

This section conducts extensive comparison experiments to compare the proposed EDPSO with several state-of-art large-scale algorithms including five holistic large-scale PSO optimizers and four state-of-the-art CCEAs. Specifically, the five holistic large-scale optimizers are TPLSO [31], SPLSO [55], LLSO [54], CSO [29], and SLPSO [55], respectively, while the four CCEAs are DECC-GDG [69], DECC-DG [74], DECC-RDG [79], and DECC-RDG2 [74], respectively. In particular, we compare EDPSO with these algorithms on the 1000-D CEC’2010 and the 1000-D CEC’2013 large-scale benchmark sets.

Table 4. Fitness comparison between EDPSO and the compared algorithms on the 1000-D CEC’2010 problems with 3 × 10⁶ fitness evaluations with respect to the median, the mean, and the standard deviation values in the fitness of the global best solutions found in 30 independent runs. The symbols “+”, “−”, and “=” above the p-value indicate that EDPSO is significantly superior to, significantly inferior to, and equivalent to the compared algorithms on the associated problems. Those numbers in bold indicates that EDPSO performs significantly better than the compared algorithms.

F	Quality	EDPSO	TPLSO	SPLSO	LLSO	CSO	SL_PSO	DECC_GDG	DECC_DG2	DECC_RDG	DECC_RDG2
F1	Median	2.59 × 10−23	1.97 × 10−18	7.86 × 10−20	2.86 × 10−22	4.63 × 10−12	7.79 × 10−18	6.57 × 100	1.95 × 10−1	2.60 × 10−3	1.05 × 10−3
	Mean	2.72 × 10−23	1.95 × 10−18	7.80 × 10−20	3.06 × 10−22	4.78 × 10−12	7.80 × 10−18	6.47 × 100	7.70 × 10−1	6.42 × 100	8.09 × 10−3
	Std	6.36 × 10−24	2.83 × 10−19	7.27 × 10−21	7.19 × 10−23	7.52 × 10−13	9.10 × 10−19	1.15 × 100	1.64 × 100	3.47 × 101	3.33 × 10−2
	p-value	-	3.02× 10−11 +	3.02 × 10−11 +	9.43 × 10−14 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F2	Median	1.10 × 103	1.10 × 103	4.47 × 102	9.72 × 102	7.52 × 103	1.94 × 103	1.40 × 103	2.99 × 103	2.98 × 103	3.00 × 103
	Mean	1.11 × 103	1.11 × 103	4.46 × 102	9.85 × 102	7.49 × 103	1.94 × 103	1.41 × 103	3.02 × 103	2.98 × 103	3.01 × 103
	Std	4.97 × 101	9.16 × 101	1.65 × 101	5.56 × 101	2.54 × 102	7.80 × 101	2.36 × 101	1.68 × 102	1.18 × 102	1.23 × 102
	p-value	-	6.63 × 10−1 =	3.01 × 10−11 −	1.16 × 10−10 −	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +
F3	Median	2.18 × 10−14	1.43 × 100	2.56 × 10−13	2.89 × 10−14	2.59 × 10−9	1.88 × 100	1.13 × 101	1.10 × 101	1.12 × 101	1.12 × 101
	Mean	2.20 × 10−14	1.44 × 100	2.52 × 10−13	2.81 × 10−14	2.60 × 10−9	1.87 × 100	1.13 × 101	1.10 × 101	1.11 × 101	1.12 × 101
	Std	9.01 × 10−16	1.29 × 10−1	1.89 × 10−14	1.48 × 10−15	1.78 × 10−10	2.48 × 10−1	6.79 × 10−1	6.60 × 10−1	6.56 × 10−1	6.74 × 10−1
	p-value	-	2.36 × 10−12 +	2.27 × 10−12 +	1.49 × 10−14 +	2.36 × 10−12 +	2.36 × 10−12 +	2.36 × 10−12 +	2.36 × 10−12 +	2.36 × 10−12 +	2.36 × 10−12 +
F4	Median	3.92 × 1011	2.74 × 1011	4.38 × 1011	4.35 × 1011	6.95 × 1011	2.83 × 1011	1.39 × 1014	1.47 × 1012	1.39 × 1012	1.50 × 1012
	Mean	4.00 × 1011	2.95 × 1011	4.34 × 1011	4.46 × 1011	7.32 × 1011	2.91 × 1011	1.43 × 1014	1.73 × 1012	1.49 × 1012	1.49 × 1012
	Std	8.10 × 1010	8.68 × 1010	8.10 × 1010	1.16 × 1011	3.48 × 1011	9.25 × 1010	3.04 × 1013	6.33 × 1011	6.42 × 1011	5.54 × 1011
	p-value	-	2.77 × 10−5 −	5.55 × 10−2 =	3.39 × 10−2 +	7.12 × 10−9 +	3.57 × 10−6 −	3.02 × 10−11 +	3.02 × 10−11 +	4.08 × 10−11 +	4.50 × 10−11 +
F5	Median	2.84 × 108	1.69 × 107	5.97 × 106	1.09 × 107	2.00 × 106	2.99 × 107	3.85 × 108	1.75 × 108	1.74 × 108	1.77 × 108
	Mean	2.82 × 108	1.67 × 107	6.37 × 106	1.15 × 107	2.49 × 106	3.09 × 107	3.83 × 108	1.78 × 108	1.71 × 108	1.74 × 108
	Std	8.94 × 106	4.65 × 106	1.73 × 106	2.58 × 106	1.32 × 106	8.58 × 106	1.56 × 107	1.96 × 107	1.88 × 107	1.62 × 107
	p-value	-	3.02 × 10−11 −	2.76 × 10−11 −	9.23 × 10−14 −	3.01 × 10−11 −	3.01 × 10−11 −	3.02 × 10−11 +	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −
F6	Median	4.00 × 10−9	2.07 × 100	1.00 × 10−8	4.00 × 10−9	8.23 × 10−7	2.14 × 101	3.55 × 105	8.86 × 100	1.06 × 101	1.06 × 101
	Mean	4.00 × 10−9	2.22 × 100	1.03 × 10−8	4.00 × 10−9	8.18 × 10−7	2.01 × 101	3.65 × 105	8.97 × 100	1.05 × 101	1.05 × 101
	Std	8.41 × 10−25	3.78 × 10−1	3.25 × 10−9	2.52 × 10−24	2.62 × 10−8	3.80 × 100	4.61 × 104	6.23 × 10−1	7.11 × 10−1	6.55 × 10−1
	p-value	-	1.21 × 10−12 +	1.21 × 10−12 +	4.79 × 10−16 +	1.21 × 10−12 +	1.21 × 10−12 +	1.21 × 10−12 +	1.21 × 10−12 +	1.21 × 10−12 +	1.21 × 10−12 +
F7	Median	9.74 × 100	9.00 × 102	4.57 × 102	8.37 × 100	2.14 × 104	6.29 × 104	2.98 × 1010	1.84 × 103	4.85 × 101	5.23 × 101
	Mean	1.21 × 101	6.02 × 103	4.82 × 102	4.22 × 101	2.15 × 104	6.57 × 104	3.16 × 1010	2.03 × 103	6.39 × 101	5.94 × 101
	Std	6.14 × 100	1.06 × 104	1.26 × 102	1.53 × 102	4.51 × 103	3.85 × 104	4.44 × 109	9.41 × 102	4.74 × 101	3.75 × 101
	p-value	-	2.03 × 10−09 +	3.02 × 10−11 +	1.54 × 10−1 =	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	1.17 × 10−9 +	2.61 × 10−10 +
F8	Median	2.41 × 107	4.74 × 105	3.11 × 107	2.33 × 107	3.86 × 107	7.69 × 106	6.98 × 108	6.55 × 102	6.56 × 10−1	4.30 × 10−1
	Mean	2.41 × 107	5.05 × 105	3.11 × 107	2.34 × 107	3.87 × 107	7.66 × 106	8.24 × 108	2.70 × 105	6.64 × 105	7.86 × 10−1
	Std	2.04 × 105	1.43 × 105	7.87 × 104	2.45 × 105	1.10 × 105	2.47 × 106	4.77 × 108	1.01 × 106	1.51 × 106	1.26 × 100
	p-value	-	3.02 × 10−11 −	3.02 × 10−11 +	2.15 × 10−13 −	3.02 × 10−11 +	3.02 × 10−11 −	3.02 × 10−11 +	3.02 × 10−11 −	2.98 × 10−11 −	3.02 × 10−11 −
F9	Median	3.99 × 107	4.27 × 107	4.66 × 107	4.55 × 107	6.72 × 107	3.33 × 107	7.56 × 108	2.24 × 108	1.76 × 108	1.80 × 108
	Mean	3.94 × 107	4.34 × 107	4.62 × 107	4.51 × 107	6.70 × 107	3.39 × 107	7.46 × 108	2.20 × 108	1.73 × 108	1.80 × 108
	Std	2.88 × 106	4.39 × 106	2.97 × 106	4.04 × 106	4.38 × 106	3.82 × 106	3.61 × 107	1.82 × 107	1.24 × 107	1.76 × 107
	p-value	-	2.39 × 10−4 +	1.29 × 10−9 +	1.43 × 10−8 +	3.02 × 10−11 +	1.03 × 10−6 −	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F10	Median	1.06 × 103	9.70 × 102	7.99 × 103	8.92 × 102	9.59 × 103	2.59 × 103	4.18 × 103	6.75 × 103	6.31 × 103	6.29 × 103
	Mean	1.08 × 103	9.77 × 102	8.01 × 103	8.87 × 102	9.59 × 103	2.82 × 103	4.16 × 103	6.73 × 103	6.32 × 103	6.30 × 103
	Std	5.90 × 101	6.92 × 101	1.17 × 102	3.77 × 101	6.49 × 101	1.30 × 103	5.47 × 101	1.01 × 102	1.14 × 102	1.03 × 102
	p-value	-	6.52 × 10−7 −	3.02 × 10−11 +	9.41 × 10−14 −	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.01 × 10−11 +
F11	Median	1.51 × 10−13	3.51 × 100	3.04 × 10−12	2.49 × 100	3.97 × 10−8	2.43 × 101	5.62 × 100	5.44 × 100	4.76 × 100	4.86 × 100
	Mean	1.49 × 10−13	3.54 × 100	3.07 × 10−12	4.60 × 100	4.01 × 10−8	2.49 × 101	5.63 × 100	5.76 × 100	4.75 × 100	4.90 × 100
	Std	4.21 × 10−15	1.27 × 100	2.73 × 10−13	4.80 × 100	2.84 × 10−9	3.25 × 100	5.50 × 10−1	7.71 × 10−1	4.88 × 10−1	3.79 × 10−1
	p-value	-	3.62 × 10−11 +	3.62 × 10−11 +	3.65 × 10−13 +	3.62 × 10−11 +	3.62 × 10−11 +	3.62 × 10−11 +	3.62 × 10−11 +	3.62 × 10−11 +	3.62 × 10−11 +
F12	Median	1.66 × 104	1.18 × 104	9.43 × 104	1.23 × 104	4.25 × 105	1.29 × 104	2.90 × 105	4.04 × 104	2.22 × 104	2.21 × 104
	Mean	1.66 × 104	1.20 × 104	9.55 × 104	1.23 × 104	4.40 × 105	1.54 × 104	2.89 × 105	3.98 × 104	2.21 × 104	2.21 × 104
	Std	1.21 × 103	1.34 × 103	6.73 × 103	1.22 × 103	6.21 × 104	7.14 × 103	1.05 × 104	2.07 × 103	1.30 × 103	1.26 × 103
	p-value	-	5.49 × 10−11 −	3.02 × 10−11 +	1.72 × 10−13 −	3.02 × 10−11 +	4.51 × 10−2 −	3.02 × 10−11 +	3.02 × 10−11 +	3.34 × 10−11 +	3.02 × 10−11 +
F13	Median	5.90 × 102	7.42 × 102	4.96 × 102	7.81 × 102	4.90 × 102	8.91 × 102	1.42 × 103	1.67 × 103	8.24 × 102	8.27 × 102
	Mean	6.18 × 102	7.57 × 102	5.53 × 102	8.21 × 102	5.55 × 102	9.88 × 102	1.47 × 103	1.84 × 103	8.23 × 102	8.49 × 102
	Std	1.62 × 102	1.16 × 102	1.67 × 102	2.63 × 102	1.77 × 102	3.87 × 102	3.57 × 102	5.12 × 102	1.38 × 102	2.00 × 102
	p-value	-	2.25 × 10−4 +	5.94 × 10−2 =	7.48 × 10−5 +	8.50 × 10−2 −	7.60 × 10−7 +	4.98 × 10−11 +	3.02 × 10−11 +	6.28 × 10−6 +	2.88 × 10−6 +
F14	Median	1.09 × 108	1.29 × 108	1.61 × 108	1.24 × 108	2.48 × 108	8.64 × 107	8.61 × 108	8.73 × 108	7.18 × 108	7.20 × 108
	Mean	1.08 × 108	1.31 × 108	1.61 × 108	1.24 × 108	2.48 × 108	8.64 × 107	8.67 × 108	8.64 × 108	7.23 × 108	7.27 × 108
	Std	6.06 × 106	1.39 × 107	7.82 × 106	7.66 × 106	1.49 × 107	7.53 × 106	3.53 × 107	3.91 × 107	3.71 × 107	3.44 × 107
	p-value	-	1.61 × 10−10 +	3.02 × 10−11 +	7.27 × 10−12 +	3.02 × 10−11 +	1.33 × 10−10 −	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F15	Median	1.05 × 104	1.01 × 104	9.93 × 103	8.33 × 102	1.01 × 104	1.12 × 104	6.75 × 103	6.74 × 103	6.55 × 103	6.58 × 103
	Mean	1.05 × 104	8.70 × 103	9.93 × 103	8.72 × 102	1.01 × 104	1.13 × 104	6.78 × 103	6.75 × 103	6.55 × 103	6.57 × 103
	Std	7.80 × 101	3.13 × 103	5.23 × 101	2.75 × 102	6.55 × 101	1.65 × 102	9.92 × 101	8.09 × 101	9.01 × 101	7.58 × 101
	p-value	-	3.02 × 10−11 −	3.02 × 10−11 −	9.42 × 10−14 −	3.01 × 10−11−	3.02 × 10−11 +	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −
F16	Median	1.76 × 10−13	1.72 × 101	4.65 × 10−12	3.98 × 100	5.69 × 10−8	2.28 × 101	3.99 × 10−4	3.91 × 10−4	1.92 × 10−5	1.88 × 10−5
	Mean	1.77 × 10−13	1.88 × 101	4.69 × 10−12	4.24 × 100	5.74 × 10−8	2.47 × 101	4.03 × 10−4	3.92 × 10−4	1.93 × 10−5	1.90 × 10−5
	Std	7.02 × 10−15	7.22 × 100	4.46 × 10−13	2.08 × 100	6.14 × 10−9	1.04 × 101	2.55 × 10−5	1.34 × 10−5	9.05 × 10−7	8.73 × 10−7
	p-value	-	2.90 × 10−11 +	1.93 × 10−3 +	2.24 × 10−12 +	1.94 × 10−3 +	2.90 × 10−11 +	1.94 × 10−3 +	1.93 × 10−3 +	2.76 × 10−11 +	2.76 × 10−11 +
F17	Median	1.42 × 105	9.72 × 104	6.92 × 105	9.15 × 104	2.21 × 106	2.97 × 104	2.65 × 105	2.65 × 105	1.99 × 105	1.97 × 105
	Mean	1.41 × 105	9.71 × 104	6.88 × 105	9.17 × 104	2.22 × 106	3.21 × 104	2.66 × 105	2.65 × 105	1.98 × 105	1.99 × 105
	Std	8.66 × 103	7.69 × 103	3.42 × 104	5.04 × 103	2.10 × 105	1.07 × 104	7.98 × 103	7.75 × 103	8.87 × 103	1.05 × 104
	p-value	-	3.02 × 10−11 −	3.02 × 10−11 +	9.43 × 10−14 −	3.02 × 10−11 +	3.02 × 10−11 −	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F18	Median	1.72 × 103	2.27 × 103	1.27 × 103	2.51 × 103	1.49 × 103	2.01 × 103	1.20 × 103	1.16 × 103	1.08 × 103	1.12 × 103
	Mean	1.76 × 103	2.33 × 103	1.37 × 103	2.56 × 103	1.69 × 103	2.19 × 103	1.18 × 103	1.15 × 103	1.07 × 103	1.12 × 103
	Std	3.45 × 102	4.22 × 102	3.77 × 102	7.20 × 102	8.08 × 102	5.17 × 102	1.38 × 102	1.24 × 102	1.09 × 102	9.95 × 101
	p-value	-	9.53 × 10−7 +	3.83 × 10−6 −	5.07 × 10−7 +	3.78 × 10−2 −	1.37 × 10−3 +	2.87 × 10−10 −	1.09 × 10−10 −	4.50 × 10−11 −	4.98 × 10−11 −
F19	Median	8.66 × 106	3.90 × 106	8.22 × 106	1.85 × 106	9.80 × 106	3.87 × 106	2.14 × 106	2.11 × 106	1.96 × 106	1.94 × 106
	Mean	8.57 × 106	3.89 × 106	8.26 × 106	1.84 × 106	9.88 × 106	3.85 × 106	2.15 × 106	2.12 × 106	1.95 × 106	1.93 × 106
	Std	5.50 × 105	2.71 × 105	4.89 × 105	8.65 × 104	5.06 × 105	5.38 × 105	1.71 × 105	1.05 × 105	7.93 × 104	9.83 × 104
	p-value	-	3.02 × 10−11 −	4.36 × 10−2 −	9.43 × 10−14 −	1.17 × 10−9 +	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −
F20	Median	1.41 × 103	2.00 × 103	9.79 × 102	1.96 × 103	1.01 × 103	1.59 × 103	5.48 × 103	5.42 × 103	4.32 × 103	4.29 × 103
	Mean	1.45 × 103	2.05 × 103	1.06 × 103	1.95 × 103	1.07 × 103	1.59 × 103	5.49 × 103	5.50 × 103	4.28 × 103	4.37 × 103
	Std	1.69 × 102	1.88 × 102	1.78 × 102	2.64 × 102	1.72 × 102	1.53 × 102	3.52 × 102	3.38 × 102	2.34 × 102	3.18 × 102
	p-value	-	6.70 × 10−11 +	1.20 × 10−8 −	9.59 × 10−12 +	3.49 × 10−9 −	1.30 × 10−3 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
w/t/l			11/1/8	12/1/7	11/1/8	15/0/5	12/0/8	17/0/3	15/0/5	15/0/5	15/0/5
Rank		3.65	4.55	4.4	3.45	6.1	5.45	7.8	6.5	7.8	7.8

and

Table 5. Fitness comparison between EDPSO and the compared algorithms on the 1000-D CEC’2013 problems with 3 × 10⁶ fitness evaluations with respect to the median, the mean, and the standard deviation values in the fitness of the global best solutions found in 30 independent runs. The symbols “+”, “−”, and “=” above the p-value indicate that EDPSO is significantly superior to, significantly inferior to, and equivalent to the compared algorithms on the associated problems. Those numbers in bold indicates that EDPSO performs significantly better than the compared algorithms.

F	Quality	EDPSO	TPLSO	SPLSO	LLSO	CSO	SL_PSO	DECC_GDG	DECC_DG2	DECC_RDG	DECC_RDG2
F1	Median	4.19 × 10−23	3.04 × 10−18	1.18 × 10−19	4.10 × 10−22	8.07 × 10−12	1.03 × 10−17	7.13 × 100	4.42 × 100	2.39 × 10−2	3.09 × 10−2
	Mean	4.43 × 10−23	3.74 × 10−18	1.21 × 10−19	4.57 × 10−22	7.94 × 10−12	1.65 × 10−17	7.59 × 100	7.11 × 100	3.87 × 10−2	1.11 × 10−1
	Std	1.26 × 10−23	1.64 × 10−18	1.21 × 10−20	1.69 × 10−22	1.23 × 10−12	3.30 × 10−17	1.03 × 100	7.73 × 100	3.97 × 10−2	2.12 × 10−1
	p-value	-	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	5.57 × 10−10 +
F2	Median	1.15 × 103	1.23 × 103	9.92 × 102	1.14 × 103	8.61 × 103	2.13 × 103	1.43 × 103	7.85 × 103	7.90 × 103	7.87 × 103
	Mean	1.15 × 103	1.23 × 103	1.11 × 103	1.15 × 103	8.59 × 103	2.14 × 103	1.44 × 103	7.92 × 103	7.79 × 103	7.78 × 103
	Std	5.03 × 101	5.85 × 101	4.74 × 102	4.54 × 101	1.84 × 102	1.48 × 102	2.18 × 101	4.07 × 102	3.64 × 102	3.65 × 102
	p-value	-	3.83 × 10−6 +	8.50 × 10−2 =	8.88 × 10−1 =	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +	3.01 × 10−11 +
F3	Median	2.16 × 101	2.16 × 101	2.16 × 101	2.16 × 101	2.16 × 101	2.16 × 101	2.15 × 101	2.15 × 101	2.14 × 101	2.15 × 101
	Mean	2.16 × 101	2.16 × 101	2.16 × 101	2.16 × 101	2.16 × 101	2.16 × 101	2.15 × 101	2.15 × 101	2.14 × 101	2.15 × 101
	Std	5.79 × 10−3	2.34 × 10−2	2.60 × 10−2	6.00 × 10−3	1.78 × 10−2	3.02 × 10−2	1.10 × 10−2	1.22 × 10−2	1.11 × 10−2	8.29 × 10−3
	p-value	-	3.22 × 10−1 =	4.78 × 10−1 =	4.33 × 10−1 =	1.20 × 10−1 =	3.50 × 10−3 +	2.98 × 10−11 −	2.98 × 10−11 −	3.00 × 10−11 −	2.96 × 10−11 −
F4	Median	6.11 × 109	4.27 × 109	9.23 × 109	6.53 × 109	1.25 × 1010	4.47 × 109	4.19 × 1011	8.21 × 1010	7.84 × 1010	6.62 × 1010
	Mean	6.05 × 109	4.28 × 109	9.60 × 109	6.68 × 109	1.37 × 1010	4.46 × 109	4.30 × 1011	8.02 × 1010	7.46 × 1010	6.92 × 1010
	Std	1.08 × 109	1.06 × 109	1.70 × 109	1.50 × 109	3.22 × 109	8.94 × 108	8.20 × 1010	2.22 × 1010	1.92 × 1010	2.32 × 1010
	p-value	-	5.60 × 10−7 −	2.15 × 10−10 +	1.02 × 10−1 =	3.02 × 10−11 +	6.05 × 10−7 −	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F5	Median	6.96 × 105	6.51 × 105	6.48 × 105	6.72 × 105	6.08 × 105	8.89 × 105	8.66 × 106	6.10 × 106	5.85 × 106	5.77 × 106
	Mean	1.97 × 106	6.70 × 105	6.40 × 105	6.73 × 105	6.09 × 105	8.98 × 105	8.70 × 106	6.07 × 106	5.78 × 106	5.71 × 106
	Std	2.70 × 106	1.02 × 105	9.32 × 104	9.47 × 104	1.04 × 105	1.27 × 105	3.10 × 105	2.21 × 105	3.91 × 105	3.58 × 105
	p-value	-	4.92 × 10−1 =	1.45 × 10−1 =	5.89 × 10−1 =	3.64 × 10−2 −	4.71 × 10−4 −	3.02 × 10−11 +	6.76 × 10−5 +	6.77 × 10−5 +	6.77 × 10−5 +
F6	Median	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106
	Mean	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106	1.06 × 106
	Std	1.07 × 103	2.34 × 103	1.83 × 103	1.17 × 103	1.09 × 103	2.23 × 103	1.38 × 103	1.43 × 103	1.25 × 103	1.23 × 103
	p-value	-	4.20 × 10−4 +	5.84 × 10−1 =	4.38 × 10−1 =	7.67 × 10−1 =	4.87 × 10−1 =	1.92 × 10−9 +	5.11 × 10−4 +	5.73 × 10−8 +	6.96 × 10−05 +
F7	Median	2.53 × 106	1.17 × 106	5.52 × 106	1.70 × 106	5.56 × 106	1.49 × 106	7.74 × 108	7.36 × 107	2.93 × 108	8.65 × 107
	Mean	2.70 × 106	1.21 × 106	5.61 × 106	1.89 × 106	5.98 × 106	1.76 × 106	7.94 × 108	7.82 × 107	3.76 × 108	8.36 × 107
	Std	1.14 × 106	4.52 × 105	2.25 × 106	1.07 × 106	3.08 × 106	1.51 × 106	1.21 × 108	2.80 × 107	2.67 × 108	2.05 × 107
	p-value	-	3.65 × 10−8 −	1.07 × 10−7 +	1.77 × 10−3 −	7.12 × 10−9 +	2.00 × 10−5 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F8	Median	1.20 × 1014	6.91 × 1013	1.59 × 1014	1.37 × 1014	2.45 × 1014	9.65 × 1013	1.74 × 1016	9.53 × 1015	6.96 × 1015	6.01 × 1015
	Mean	1.19 × 1014	7.19 × 1013	1.59 × 1014	1.38 × 1014	2.50 × 1014	1.11 × 1014	1.78 × 1016	9.38 × 1015	7.03 × 1015	6.58 × 1015
	Std	3.47 × 1013	3.55 × 1013	2.76 × 1013	3.45 × 1013	8.60 × 1013	5.49 × 1013	5.51 × 1015	2.73 × 1015	1.57 × 1015	2.08 × 1015
	p-value	-	3.52 × 10−7 −	1.25 × 10−5 +	1.63 × 10−2 +	5.46 × 10−9 +	1.22 × 10−1 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F9	Median	1.13 × 108	4.33 × 107	7.93 × 107	1.13 × 108	6.11 × 107	8.10 × 107	5.73 × 108	5.58 × 108	5.43 × 108	5.36 × 108
	Mean	1.21 × 108	4.23 × 107	8.27 × 107	1.47 × 108	6.28 × 107	8.11 × 107	5.66 × 108	5.62 × 108	5.42 × 108	5.34 × 108
	Std	3.29 × 107	6.71 × 106	2.18 × 107	1.61 × 108	1.31 × 107	1.09 × 107	3.06 × 107	3.28 × 107	3.02 × 107	2.38 × 107
	p-value	-	3.02 × 10−11 −	3.09 × 10−6 −	8.07 × 10−1 =	2.15 × 10−10 −	9.06 × 10−8 −	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F10	Median	9.41 × 107	9.42 × 107	9.40 × 107	9.42 × 107	9.41 × 107	9.37 × 107	9.47 × 107	9.46 × 107	9.47 × 107	9.46 × 107
	Mean	9.41 × 107	9.42 × 107	9.40 × 107	9.43 × 107	9.41 × 107	9.29 × 107	9.46 × 107	9.46 × 107	9.46 × 107	9.46 × 107
	Std	2.69 × 105	2.77 × 105	2.91 × 105	3.26 × 105	2.68 × 105	1.55 × 106	2.03 × 105	2.51 × 105	1.84 × 105	2.47 × 105
	p-value	-	9.93 × 10−02 =	1.26 × 10−1 =	3.27 × 10−2 +	7.06 × 10−1 =	1.41 × 10−4 −	3.16 × 10−10 +	2.44 × 10−9 +	9.91 × 10−11 +	5.96 × 10−9 +
F11	Median	9.26 × 1011	1.79 × 108	9.23 × 1011	9.27 × 1011	9.35 × 1011	9.39 × 1011	6.93 × 108	2.09 × 1010	5.84 × 108	1.43 × 1010
	Mean	9.29 × 1011	1.78 × 108	9.29 × 1011	9.31 × 1011	9.30 × 1011	9.36 × 1011	7.01 × 108	2.68 × 1010	5.81 × 108	1.52 × 1010
	Std	1.00 × 1010	5.06 × 107	1.04 × 1010	1.08 × 1010	9.36 × 109	8.27 × 109	1.09 × 108	1.48 × 1010	8.79 × 107	7.61 × 109
	p-value	-	3.02 × 10−11 −	8.42 × 10−1 =	5.01 × 10−1 −	9.12 × 10−1 =	1.17 × 10−2 +	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −
F12	Median	1.43 × 103	2.03 × 103	1.03 × 103	1.82 × 103	1.05 × 103	1.76 × 103	5.56 × 103	5.51 × 103	4.31 × 103	4.32 × 103
	Mean	1.43 × 103	2.04 × 103	1.06 × 103	1.84 × 103	1.09 × 103	1.78 × 103	5.54 × 103	5.63 × 103	4.37 × 103	4.33 × 103
	Std	8.89 × 101	2.38 × 102	5.91 × 101	1.62 × 102	8.43 × 101	1.74 × 102	3.77 × 102	7.69 × 102	3.24 × 102	2.68 × 102
	p-value	-	3.02 × 10−11 +	4.45 × 10−11 −	5.49 × 10−11 +	6.69 × 10−11 −	6.12 × 10−10 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +
F13	Median	4.28 × 108	1.84 × 108	1.22 × 109	3.07 × 108	7.21 × 108	4.03 × 108	1.59 × 109	1.47 × 109	2.97 × 109	7.15 × 108
	Mean	4.40 × 108	2.12 × 108	1.24 × 109	3.45 × 108	7.66 × 108	6.26 × 108	1.57 × 109	1.54 × 109	3.08 × 109	7.23 × 108
	Std	1.50 × 108	1.17 × 108	5.06 × 108	1.44 × 108	2.87 × 108	8.23 × 108	3.45 × 108	3.47 × 108	8.07 × 108	1.58 × 108
	p-value	-	1.87 × 10−7 −	6.12 × 10−10 +	1.27 × 10−2 −	7.09 × 10−8 +	7.06 × 10−1 =	3.02 × 10−11 +	3.02 × 10−11 +	3.02 × 10−11 +	9.83 × 10−8 +
F14	Median	2.01 × 108	5.81 × 107	5.19 × 109	9.27 × 107	2.92 × 109	1.56 × 108	4.54 × 109	4.83 × 109	2.22 × 109	3.09 × 109
	Mean	3.67 × 108	5.98 × 107	8.36 × 109	1.61 × 108	3.74 × 109	2.71 × 108	5.45 × 109	4.71 × 109	2.87 × 109	3.43 × 109
	Std	4.23 × 108	1.28 × 107	6.66 × 109	2.30 × 108	3.35 × 109	2.36 × 108	3.96 × 109	1.80 × 109	1.90 × 109	2.07 × 109
	p-value	-	1.35 × 10−10 −	4.46 × 10−11 +	9.45 × 10−5 −	4.94 × 10−11 +	2.92 × 10−1 =	8.20 × 10−11 +	8.20 × 10−11 +	3.29 × 10−10 +	1.49 × 10−10 +
F15	Median	4.53 × 107	1.24 × 107	4.13 × 107	4.63 × 106	7.62 × 107	6.06 × 107	8.70 × 106	8.92 × 106	7.89 × 106	8.03 × 106
	Mean	4.70 × 107	1.23 × 107	4.19 × 107	4.63 × 106	7.64 × 107	6.10 × 107	9.05 × 106	9.08 × 106	8.05 × 106	8.16 × 106
	Std	7.77 × 106	9.89 × 105	3.55 × 106	3.38 × 105	6.06 × 106	6.32 × 106	9.14 × 105	9.03 × 105	1.01 × 106	1.02 × 106
	p-value	-	3.02 × 10−11 −	5.57 × 10−3 −	3.02 × 10−11 −	3.69 × 10−11 +	4.31 × 10−8 +	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −	3.02 × 10−11 −
w/t/l			4/3/8	6/6/3	4/6/5	8/4/3	8/3/4	12/0/3	12/0/3	12/0/3	12/0/3
Rank		4.07	3.53	4.73	4.00	5.53	4.40	8.20	7.60	6.80	6.13

show the fitness comparison results between EDPSO and the nine compared algorithms on the CEC’2010 and the CEC’2013 benchmark sets, respectively. In these two tables, the symbol “+” above the p-value indicates that the proposed EDPSO is significantly superior to the associated compared algorithms on the corresponding problems, and the symbol “−” means that EDPSO is significantly inferior to the associated compared algorithms on the related problems, while the symbol “=” denotes that EDPSO achieves equivalent performance with the compared algorithms on the associated problems. Accordingly, “w/t/l” in the second to last rows of the two tables count the numbers of “+”, “=”, and “–”, respectively. In addition, the last rows of the two tables present the averaged ranks of all algorithms, which are obtained from the Friedman test.

From Table 4, the comparison results between EDPSO and the nine compared algorithms on the 20 1000-D CEC’2010 benchmark problems are summarized as follows:

(1)

From the perspective of the averaged ranks (shown in the last row) obtained by the Friedman test, EDPSO and LLSO achieve much smaller rank values than the other eight compared algorithms. This demonstrates that the proposed EDPSO and LLSO achieve significantly better performance on the CEC’2010 benchmark set than the other eight compared algorithms.

(2)

In view of the results (shown in the second to last row) obtained from the Wilcoxon rank-sum test, it is intuitively found that the proposed EDPSO is significantly better than the nine compared algorithms on at least 11 problems, and shows inferiority to them on at most eight problems. In particular, compared with the five holistic large-scale PSOs (namely TPLSO, SPLSO, LLSO, CSO, and SL-PSO), EDPSO shows significant dominance to them on 11, 12, 11, 15, and 12 problems, respectively, and only displays inferiority to them on eight, seven, eight, five, and eight problems, respectively. In comparison with the four CCEAs (namely, DECC-GDG, DECC-DG2, DECC-RDG, and DECC-RDG2), EDPSO significantly outperforms DECC-GDG on 17 problems, and significantly beats DECC-DG2, DECC-RDG, and DECC-RDG2 on all 15 problems.

From Table 5, we can draw the following conclusions with respect to the comparison results between the proposed EDPSO and the nine state-of-the-art compared methods on the 15 1000-D CEC’2013 benchmark problems:

(1)

In terms of the average rank values obtained from the Friedman test, the proposed EDPSO achieves a much smaller rank value than seven compared algorithms. This indicates that the proposed EDPSO achieves much better overall performance than them on such a difficult benchmark set. In particular, EDPSO achieves considerably similar performance with LLSO, but obtains slightly inferior performance to TPLSO.

(2)

With respect to the results obtained from the Wilcoxon rank-sum test, in comparison with the five holistic large-scale PSO variants, EDPSO significantly dominates SPLSO, CSO, and SL-PSO six, eight, and eight problems, respectively, and only exhibits inferiority to them in three, three, and four problems, respectively. Compared with LLSO, EDPSO achieves very competitive performance with it. However, in comparison with TPLSO, EDPSO shows slightly inferiority on this benchmark set. Competed with the four CCEAs, EDPSO presents significant superiority to them all on 12 problems and displays inferiority to them all on three problems.

The above experiments have demonstrated the effectiveness of the proposed EDPSO in solving high-dimensional problems. To further demonstrate its efficiency, we conduct experiments on the CEC’2010 and the CEC’2013 benchmark sets to investigate the convergence behavior comparisons between EDPSO and the compared algorithms. In these experiments, we set the maximum number of fitness evaluations as 5 × 10⁶ for all algorithms. Then, we record the global best fitness every 5 × 10⁵ fitness evaluations. Figure 2 and Figure 3 display the convergence behavior comparisons between EDPSO and the nine compared algorithms on the CEC’2010 and the CEC’2013 benchmark sets, respectively.

From Figure 2, at first glance, we find that EDPSO achieves clearly better performance than the nine compared algorithms in terms of both convergence speed and solution quality on three problems (F₁, F₁₁, and F₁₆). In particular, EDPSO could finally find the true global optimum of F₁. Taking deep observation, it is found that on the whole CEC’2010 benchmark set, the proposed EDPSO achieves highly competitive or even much better performance than most of the compared algorithms on most problems with respect to both the convergence speed and the solution quality. This implicitly demonstrates that the proposed EDPSO is efficient when solving high-dimensional problems.

From Figure 3, the convergence comparison results between the proposed EDPSO and the compared algorithms on the CEC’2013 show that the proposed EDPSO still presents considerably competitive or even great superiority to most of the compared algorithms on most problems in terms of both the convergence speed and the solution quality. This further demonstrates that the proposed EDPSO is efficient when solving complicated high-dimensional problems.

The above experiments on the two benchmark sets have demonstrated the effectiveness and efficiency of the proposed EDPSO. The superiority of EDPSO in solving high-dimensional problems mainly benefits from the proposed elite-directed learning strategy, which treats particles in different layers differently and lets particles in different layers learn from different numbers of predominant particles in higher layers. With this strategy, the learning effectiveness and the learning diversity of particles are largely improved, and the algorithm could compromise the search diversification and intensification well at the swarm level and the particle level to explore and exploit the solution space properly to obtain satisfactory performance.

4.3. Effectiveness of the Additional Archive

In this section, we conduct experiments to verify the usefulness of the additional archive. To this end, we first develop two versions of EDPSO. First, we remove the archive in EDPSO, leading to a variant of EDPSO, which we name as “EDPSO-WA”. Second, to demonstrate the effectiveness of using predominant individuals in the archive that are better than the best in the third layer, we replace this strategy by randomly choosing individuals in the archive along with the first two layers to direct the update of particles in the third layer. As a result, a new variant of EDPSO, which we name “EDPSO-ARand”, is developed. Then, we conduct experiments on the CEC’2010 benchmark set to compare the three different versions of EDPSO.

Table 6. Fitness comparison between EDPSO with and without the archive on the 1000-D CEC’2010 problems with 3 × 10⁶ fitness evaluations with respect to the mean fitness of the global best solutions found in 30 independent runs. The best results are highlighted in bold.

F	EDPSO	EDPSO-WA	EDPSO-ARand
F1	2.72 × 10−23	2.25 × 10−20	3.13 × 10−18
F2	1.11 × 103	1.25 × 103	7.58 × 102
F3	2.20 × 10−14	6.82 × 10−14	1.46 × 10−12
F4	4.00 × 1011	7.43 × 1011	3.36 × 1011
F5	2.82 × 108	2.91 × 108	2.73 × 108
F6	4.00 × 10−9	4.31 × 10−9	4.76 × 10−9
F7	1.21 × 101	1.32 × 104	1.99 × 104
F8	2.41 × 107	3.53 × 107	3.19 × 107
F9	3.94 × 107	7.88 × 107	1.85 × 107
F10	1.08 × 103	1.23 × 103	9.40 × 103
F11	1.49 × 10−13	5.82 × 100	1.68 × 10−11
F12	1.66 × 104	6.79 × 104	2.22 × 105
F13	6.18 × 102	8.85 × 102	4.78 × 102
F14	1.08 × 108	2.21 × 108	6.59 × 107
F15	1.05 × 104	1.06 × 104	9.74 × 103
F16	3.44 × 10−1	8.39 × 10−1	2.85 × 10−11
F17	1.41 × 105	3.19 × 105	2.07 × 106
F18	1.76 × 103	3.40 × 103	1.02 × 103
F19	8.57 × 106	9.96 × 106	1.00 × 107
F20	1.45 × 103	2.07 × 103	9.84 × 102
Rank	1.52	2.57	1.90

presents the comparison results with the best results in bold.

From Table 6, we can obtain the following findings. (1) With respect to the results of the Friedman test, the proposed EDPSO achieves the smallest rank among the three versions of EDPSO. In addition, from the perspective of the number of problems where the associated algorithm obtains the best results, we find the proposed EDPSO obtains the best results on 10 problems. These observations demonstrate the effectiveness of the additional archive and the way of using this archive. (2) Further, compared with EDPSO-WA, both the proposed EDPSO and EDPSO-ARand achieve much better performance. This demonstrates the usefulness of the additional archive. In comparison with EDPSO-ARand, the proposed EDPSO performs much better. This demonstrates the usefulness of the way of using predominant individuals in the archive to direct the update of particles in the third layer.

To summarize, the additional archive is helpful for EDPSO to achieve promising performance. This is because it introduces more candidate exemplars for particles in the third layer without destroying the convergence of the swarm. In this way, the learning diversity of particles is further promoted, which is beneficial for the swarm to explore the solution space and escape from local regions.

5. Conclusions

This paper has proposed an elite-directed particle swarm optimization with historical information (EDPSO) to tackle large-scale optimization problems. By taking inspiration from the “Pareto Principle”, which is also known as the 80-20 rule, we first partition the swarm into three layers with the first layer containing the top best 20% of particles and the other two layers of the same size consisting of the remaining 80% of particles. Then, particles in the third layer are updated by learning from two different exemplars randomly selected from those in the first two layers, and particles in the second layer are updated by learning from two different exemplars randomly selected from the first layer. To preserve the valuable evolutionary information, particles in the first layer are not updated and directly enter the next generation. To make full use of potentially valuable historical information, we additionally maintain an archive to store the obsolete elites in the first layer and then introduce them to the learning of particles in the third layer. With the above two techniques, particles in the last two layers could learn from different numbers of predominant candidates, and therefore the learning effectiveness and the learning diversity of particles are expectedly largely improved. By means of the proposed elite-directed learning strategy, EDPSO could compromise exploration and exploitation well at both the particle level and the swarm level to search high-dimensional space, which contributes to its promising performance solving complex large-scale optimization problems.

Extensive experiments have been conducted on the widely used CEC’2010 and CEC’2013 large-scale benchmark sets to validate the effectiveness of the proposed EDPSO. Experimental results have demonstrated that the proposed EDPSO could consistently achieve much better performance than the compared peer large-scale optimizers on the two benchmark sets. This verifies that EDPSO is promising for solving large-scale optimization problems in real-world applications.

However, in EDPSO, the control parameter ϕ is currently fixed and needs fine-tuning, which requires a lot of effort. To alleviate this issue, in the future, we will concentrate on devising an adaptive adjustment scheme for ϕ to eliminate its sensitivity based on the evolution state of the swarm and the evolutionary information of particles. In addition, to promote the application of the proposed EDPSO, in the future, we will also focus on utilizing EDPSO to solve real-world optimization problems.