Multiple Learning Strategies and a Modified Dynamic Multiswarm Particle Swarm Optimization Algorithm with a Master Slave Structure

Abstract

It is a challenge for the particle swarm optimization algorithm to effectively control population diversity and select and design efficient learning models. To aid in this process, in this paper, we propose multiple learning strategies and a modified dynamic multiswarm particle swarm optimization with a master slave structure (MLDMS-PSO). First, a dynamic multiswarm strategy with a master–slave structure and a swarm reduction strategy was introduced to dynamically update the subswarm so that the population could maintain better diversity and more exploration abilities in the early stage and achieve better exploitation abilities in the later stage of the evolution. Second, three different particle updating strategies including a modified comprehensive learning (MCL) strategy, a united learning (UL) strategy, and a local dimension learning (LDL) strategy were introduced. The different learning strategies captured different swarm information and the three learning strategies cooperated with each other to obtain more abundant population information to help the particles effectively evolve. Finally, a multiple learning model selection mechanism with reward and punishment factors was designed to manage the three learning strategies so that the particles could select more advantageous evolutionary strategies for different fitness landscapes and improve their evolutionary efficiency. In addition, the results of the comparison between MLDMS-PSO and the other nine excellent PSOs on the CEC2017 test suite showed that MLDMS-PSO achieved an excellent performance on different types of functions, contributing to a higher accuracy and a better performance.

1. Introduction

Many real-world optimization problems such as classification problems, nonlinear processes, path selection, scheduling problems, structural designs, and text mining may have multimode, non-convex, nonlinear, discontinuous, and non-differentiable characteristics, which makes these problems extremely complex and difficult to solve by traditional methods [1]. To solve these problems better and more effectively, many researchers have developed population-based nature-inspired search optimization algorithms such as genetic algorithms (GAs) [2], particle swarm optimizations (PSOs) [3], ant colony optimizations (ACOs) [4], gravitation search algorithms (GSAs) [5], differential evolutions (DEs) [6], and distribution estimation algorithms (EDAs) [7], proposing various population organization patterns and learning mechanisms, cleverly utilizing individual historical knowledge to help population evolution. These population-based algorithms are capable of solving most optimization problems and are easy to implement. Due to the random initialization of the population and the introduction of random numbers into the search mechanism, their dependence on the initial scheme and gradient information is lessened to a great extent.

Among the population-based algorithms, PSO [3] was proposed by Kennedy and Eberhart in 1995 based on research on the predation behavior of birds. It finds the best solution through information sharing and cooperation among the individuals in the population. Because of its simplicity, a few parameters adjusted, and its high computational efficiency, PSO has successfully attracted the extensive attention of engineers and scholars in recent decades. The classical PSO constructs a learning mechanism in which each particle realizes its own evolution through self-cognition and social cognition. Self-cognition makes it maintain its own historical experience, and social cognition makes the individuals cooperate and share information. This learning mechanism enables particles to continuously mine better positions from the search space and guide the subsequent search behavior of particles, which gives PSO a strong ability to solve complex problems.

The learning model of PSO, that is, the information sharing mode among particles, largely determines the ability of the algorithm to solve complex problems. Generally, it is an effective way to enhance the algorithm ability to fully mine and use the effective information in the population to build an efficient learning model. In the classical PSO, particles perform the learning process by measuring the fitness and selecting the global best-so-far particles or the personal best experience as a learning model [8,9]. When optimizing the single-mode function, this learning model can help particles quickly find the global optimum. However, PSO may lose its diversity and fall into a premature convergence because the overall learning mode makes particles gradually move to one dominating candidate in the population until convergence, especially when solving nonseparable and high-dimensional optimization problems or optimization problems with many local optimizations [10].

For the population-based evolutionary algorithm, maintaining the diversity of the population gathers increasingly effective evolutionary information. According to the characteristics of population evolution, different evolutionary information is selected as a particle learning example, which balances the exploration and development and guides the population to evolve effectively. For instance, dynamic multiswarm PSO (DMS-PSO) [11] allocates all particles into multiple subswarms. The individuals in each subswarm receive different social information so that each subswarm has different evolutionary paths, and information among the subswarms is shared through population reorganization, which effectively maintains the diversity and makes it perform well on complex multimode problems. However, DMS-PSO is not conducive to the evolution of some subswarms because it adopts the same learning template for different subswarms. A heterogeneous comprehensive learning and dynamic multiswarm PSO (HCLDMS-PSO) [12] is presented, where a comprehensive learning (CL) strategy was used as the exploitation subswarm learning model and an improved DMS strategy was designed to structure the exploration subswarm learning model. The two subswarms rely on the different learning exemplars to update independently to ensure that the population does not lose its diversity.

Exploration and exploitation are two important stages for the evolution process of population-based algorithms [13,14]. In the exploration stage, to seek the global optimal solution, individuals need to reach different regional positions as much as possible, and better diversity is obtained at the population level. In the exploitation stage, particles search in a small area for the sake of obtaining a high-precision optimal solution faster, which can effectively improve the local search ability of the algorithm but sacrifices the diversity of the population and increases the risk of premature convergence. Therefore, the probability of the premature convergence can be significantly reduced by improving the diversity of the population. The multiswarm strategy divides the entire population into several subswarms. Individuals within a subswarm cooperate and share information with other subswarms to produce new offspring individuals, which makes each subswarm evolve along different paths and effectively maintains the diversity of the population during the evolution. To date, the multiswarm strategy has been widely used in population-based evolutionary algorithms because it has natural advantages in maintaining population diversity. Various population division strategies have been proposed such as stochastic division, clustering, fitness sharing, etc., in which the algorithm extends the learning beyond the dominant individuals by improving the probability of poor individuals entering the next generation. Their common starting point is to maintain the diversity of the population.

In addition, considering that the update mode of all individuals is the same for the algorithm of a single learning model, the population information cannot be fully used to guide individual evolution, which cannot intelligently deal with different complex situations. In other words, when the algorithm is in different evolutionary stages, or is used to solve different types of optimization problems, the guiding effects of the different learning models are quite different. Therefore, some scholars have begun to explore PS based on a variety of different learning models. For instance, in the triple archives PSO (TAPSO) [15], three different learning models, the Diffident Model, the Mild Model, and the Confident Model, are provided to update particles so that the particles in the population show different search behaviors, meeting the needs of the algorithm in its different evolutionary stages. In the multiswarm PSO with dynamic learning strategy (PSO-DLS) [16], particles take the local optimal position in their subswarms (focusing on exploitation) and a united local best position among subswarms (focusing on exploration), as examples. During the search process, the population can maintain diversity without being affected by the global best position of all the dominant particles.

In summary, multiple population structures can offer significant diversity at the population level, and excellent learning strategies can more extensively mine population historical knowledge, which are effective in helping find the global optimum, especially for problems with multiple local optimal solutions. However, most studies have a fixed number of populations. If the population size is large, it helps with the initial exploration of the algorithm, but limits the local development ability in the later stages of evolution, and vice versa. Meanwhile, the rational management of multiple learning strategies to match the evolutionary characteristics of the population is also an effective way to improve the overall performance of the algorithm. Motivated by this observation, we propose multiple learning strategies and a modified dynamic multiswarm PSO with a master–slave structure, called MLDMS-PSO. Specifically, the characteristics of MLDMS-PSO are as follows.

Adopting a multiswarm strategy is an effective means to improve population diversity. In this paper, a modified dynamic multiswarm (MDMS) strategy with a master–slave structure is proposed. Specifically, among the superior particles of the population, we selected the particle combinations with a more uniform spatial distribution as the master particles. With the master particles as the center, the subswarms were constructed. At the same time, the number reduction strategy of the subswarm was adopted to strengthen the local exploitation ability of the algorithm in its late evolution. Three learning strategies were adopted to update the particle positions. Respectively, a modified comprehensive learning (MCL) strategy is conducive to the diversity of the population; a local dimension learning (LDL) strategy can help better local exploitation; a united learning (UL) strategy was used to balance the MCL and LDL. In addition, to further exert the advantages of the three learning strategies, a probability function with a reward and punishment mechanism was designed to manage the three learning strategies, strengthen the competition within the subswarms and the collaboration between subswarms, and improve the search efficiency of the algorithm. Finally, a large number of experiments was carried out on 29 benchmark functions from the CEC2017 test suite. The experimental results, supported by the nonparametric Wilcoxon rank-sum test and Friedman test, consistently demonstrated the that MLDMS-PSO could effectively balance exploration and exploitation, clearly improve the convergence accuracy and reliability, and was able to provide a better and more consistent performance than all comparison algorithms on the majority of the benchmark functions.

The rest of this paper is arranged as follows. Section 2 shows the standard PSO and reviews some different types of PSOs. The proposed MLDMS-PSO is detailed in Section 3. In Section 4, extensive experiments are carried out to test the MLDMS-SSO, and its performance is analyzed and discussed in depth. Finally, the conclusions are presented in Section 5.

2. Review of the Standard PSO and Its Variants

2.1. Standard PSO

As a kind of population intelligent optimization algorithm, PSO is generally used because of its excellent performance in solving continuous optimization problems. It is assumed that the population contains M particles in the D-dimensional search space, and each particle is composed of two kinds of information, the particle position, and velocity, in which the particle position vector, ${\mathit{x}}_i={(x_{i,1},x_{i,2},\dots ,x_{i,D})}^T$, represents a solution in the search space, and the particle velocity vector, ${\mathit{v}}_i={(v_{i,1},v_{i,2},\dots ,v_{i,D})}^T$, indicates the search direction of the particle. Particle x_i moves at speed v_i in the search space. Due to the influence of inertia, it is more inclined to maintain its own speed and can learn from its personal best experience Pbest_i and population best experience Gbest to improve the v_i. In the initialization phase, each particle is given a random initial position and an initial velocity. After that, the update equations of v_i and x_i are as follows:

$$v_{i,j}^{(t+1)}=w{v}_{i,j}^{(t)}+c_1{r}_{1,j}(Pbes{t}_{i,j}^{(t)}-x_{i,j}^{(t)})+c_2{r}_{2,j}(Gbes{t}_j^{(t)}-x_{i,j}^{(t)})$$

(1)

$$x_{i,j}^{(t+1)}=x_{i,j}^{(t)}+v_{i,j}^{(t+1)}$$

(2)

where i = 1, 2,⋯, M, j = 1, 2,⋯, D. w is an inertia weight, c₁ and c₂ are the acceleration coefficients, and r_1,j and r_2,j are two uniformly distributed random numbers independently generated within [0, 1] for the jth dimension. t is the number of iterations. The velocity and position of ${\mathit{x}}_i^{(t)}$ are updated with Equations (1) and (2) and achieve the new position ${\mathit{x}}_i^{(t+1)}$ of particle i. If the fitness of ${\mathit{x}}_i^{(t+1)}$ ($\mathit{f}({\mathit{x}}_i^{(t+1)})$) is better than $Pbes{t}_i^{(t)}$, $Pbes{t}_i^{(t)}$ is replaced by $\mathit{f}({\mathit{x}}_i^{(t+1)})$. Otherwise, $Pbes{t}_i^{(t)}$ will remain unchanged. Similarly, Gbest can be updated until the termination conditions are met.

2.2. Some Variants of PSO

In recent decades, the real-world problems that need to be solved have become increasingly complex. The classical PSO algorithm often shows premature convergence and a poor global search ability. To improve the performance of the classical PSO, some researchers and engineers have proposed plenty of PSO variants. Depending on the objectives to be dealt with, most PSO variants can be divided into four types: parameter tunings, domain topology adjustments, learning strategies, and hybrid strategies, which are briefly reviewed here.

(1) 

Parameter tuning

Generally, the parameter control of a PSO mainly focuses on the inertia weight (w) and acceleration coefficients (c₁ and c₂). The inertia weight, used to maintain the direction of particle movement, is a critical parameter of a standard PSO. Reducing the inertia weight over time can significantly improve the performance of the PSO [17]. Now, many PSOs still use this update strategy to linearly reduce w from 0.9 to 0.4 with the progress of iteration [18,19,20,21]. It has been found through experiments that with a smaller w, the particles move at a smaller pace, which is conducive to exploitation, while with a larger w, particles can appear at a farther place each time, which is more helpful to exploration. However, considering the complexity of the PSO search process, some researchers have introduced nonlinear-varying strategies to assign inertia weights, which is able to endow particles with diverse search behaviors [22,23,24,25,26,27].

Although the above-mentioned inertia weight updating methods improve the performance of the PSO to a certain extent, they are not good enough to match between the parameter adjustments and the search process due to the lack of appropriately used evolutionary state information in the iterative process. To obtain a more satisfactory adjustment for the parameters, the inertia weight can be updated based on some feedback parameters. For instance, in [25], the success rate of the swarm was used as a feedback parameter to adaptively adjust the inertia weight, which can configure a more appropriate value of the inertia weight according to the particles’ situation in the search space. In an adaptive PSO (APSO) [28], a systematic adaptation scheme is used to control w, c₁, and c₂. This scheme is not only based on an evolutionary state estimation, but also considers the influence of these parameters in different evolutionary states. In the artificial multiswarm PSO (AMPSO) [29], an adaptive inertia weight strategy based on a hybrid of position diversity is matched to different subswarms to better balance exploration and exploitation and improve the performance of the algorithm. What these methods have in common is that they can deeply mine the fitness, speed, position, and other information of the particles, which can be effectively used to adjust their own parameters during the evolution process.

(2) 

Neighborhood topology adjustments

The PSO particles with different neighborhood topologies are connected together through different structures. In these structures, the particles have different rules for communicating and sharing information, which can affect the diffusion speed of the information in the population and drive the whole population’s evolution. Therefore, neighborhood topology adjustments can also effectively improve the performance of PSO. Many neighborhood topologies have been proposed such as the star [3], ring [30], wheel [31], and Von Neumann topologies [32]. These foregoing neighborhood topologies belong to static topologies and the pattern of information acquisition is fixed, which contradicts the distinct requirements at the different evolutionary stages. Therefore, many dynamic topologies have been developed in which the neighborhood can be refreshed and regrouped depending on the particles’ parameters in the evolution process. In [33], a dynamic neighborhood that is introduced contains only a few particles during the initial stages of the optimization. With the increase in generations, the number of particles in the neighborhood gradually increases until all particles are included. This enhances the exploration abilities of the algorithm in the earlier stage and makes the algorithm focus more on exploitation at the end of the PSO run. Furthermore, in [26], the nonparametric PSO (NP-PSO) constructs two different topologies, making it perform global and local search over the search space with a fast convergence speed. In the dynamic neighborhood learning PSO (DNLPSO) [34], the exemplar particle is selected from a neighborhood dynamically updated at certain intervals, which is conducive to preserving the diversity of the population to discourage premature convergence.

The multipopulation mechanism has received extensive attention because of its advantages in maintaining population diversity. In the DMS-PSO [11] and the dynamic multiswarm global PSO (DMS-GPSO) [35], the entire population is divided into many small swarms and various regrouping schemes and information interaction methods have been designed to perform dynamic updating of these swarms. In addition, in the multiswarm ensemble PSO (MPEPSO) [36], all particles are divided into four subswarms including three indicator subswarms and one reward subswarm. AMPSO [29] contains three subswarms, namely, an exploration swarm, an artificial exploitation swarm, and an artificial convergence swarm. In the above two algorithms, the subswarms update their velocities through different strategies, with the intention of balancing exploration and exploitation.

(3) 

Learning strategy

In PSO, the learning strategy plays a very important role because it is one of the key factors determining the particle search direction. In the global PSO (GPSO) [17], the best historical experience of the whole population is selected as an example of a particle learning. Similarly, a particle selects its local neighbors’ best positions as its learning exemplar in the local PSO (LPSO) [30]. In OLPSO [19], the method of orthogonal experimental design (OED) is used to construct the particles’ learning exemplar and the combination of the historical best experience of particles and its neighborhood’s best historical guide particles to move toward the global optimal region. In the comprehensive learning PSO (CLPSO) [21], any dimension of a particle position can learn from its historical best position or other particles with better positions through a comprehensive learning strategy, while retaining the ability to learn from the global optimal experience. Similarly, in the two-swarm learning PSO (TSLPSO) [20], a novel dimensional learning (NDL) strategy is constructed by learning from the corresponding dimensions of the population best solution through each dimension of the particle’s personal best experience to discover and integrate promising information, which focuses more on local search. In CCPSO-ISM [37], an information sharing mechanism (ISM), and a kind of competitive and cooperative operator (CC), are proposed to enhance the PSO performance. The ISM needs to keep the best information of each particle so that other particles can read the shared information, which also realizes full communication between the particles in the population and each other. The CC operation provides a more efficient way to utilize this shared information. In PSO-DLS [16], a learning strategy focusing on exploration is proposed in which particles can learn from a united local best position and can also obtain information from other subswarms to improve themselves. In [8,28], an elite learning strategy guides the best knowledge of each particle to the global optimum by effectively sharing the best experience of the dominant particles when the evolution process is in a convergent state. However, in many cases, researchers have preferred to integrate multiple learning strategies to effectively guide particles toward a global optimization to meet the needs of distinct information at the different evolutionary stages.

(4) 

Hybridization of PSO with other algorithms

Some scholars have improved the performance of PSO by hybridizing the evolutionary mechanisms of other meta heuristic algorithms such as ACO, GA, DE, and GSA. In the genetic learning PSO (GL-PSO) [38], genetic operators are used to generate exemplars with high quality and rich diversity by performing crossover, mutation, and selection on the historical information of the particles. One of the advantages of DE is that it maintains population diversity. In [9,39,40], the differential evolution strategy was introduced into the particle swarm optimization algorithm, which could effectively improve the global optimization ability of a PSO. A novel hybrid PSO and GSA (HPSO-GSA) [41] uses coevolutionary technology, which updates particle positions simultaneously with PSO velocity and GSA acceleration, to seek an effective balance between exploration and exploitation. In the ant particle optimization algorithm (HAP) [42], the PSO and ACO work and generate the best solution, taken as the global best solution, which is used as an important basis for selecting the position of next generation particles and ants. In addition, various other algorithms are hybridized with PSO to improve the overall performance such as HPPSO [43] (PSO and POA), HFPSO [44] (Firefly and PSO), PSO-BOA [45], PSOGWO [46], and so on.

3. The Proposed Method

Based on the excellent research results of previous researchers, we proposed the MLDMS-PSO algorithm to improve the performance of the particle swarm optimization algorithm, especially in dealing with more complex multimodal function problems. An MDMS strategy with a master–slave structure helps the population maintain better diversity, and multimodel learning strategies can better utilize historical knowledge in evolution. In addition, an efficient learning strategy management mechanism was established to schedule learning strategies more reasonably based on the evolutionary stage of the population, thereby significantly enhancing the overall performance of the algorithm. The steps are shown as a flowchart of the MLDMS-PSO in Figure 1.

3.1. The Proposed MDMS Strategy

In the process of iteration, PSO continuously discards the particles with poor positions and retains the particles with better positions to achieve the goal of the population’s evolution toward a better position and the search for an optimal position. Therefore, the superior particles lead each subswarm in a dynamic multiswarm strategy that can effectively avoid the population falling into a local optimum and improve the search efficiency of the population. Based on this, we propose a modified dynamic multiswarm strategy with a master–slave structure in which the subswarm is constructed with the superior particle as the center.

(1)

A multiswarm segmentation scheme with a master–slave structure

Assuming that the population is composed of M particles, it is divided into sn subswarms $\{\mathit{subpo}{\mathit{p}}_1,\mathit{subpo}{\mathit{p}}_2,\dots ,\mathit{subpo}{\mathit{p}}_{sn}\}$, and each subswarm contains sm particles, $M={\displaystyle \sum _{si=1}^{sn}s{m}_{si}}$, including a master particle and (sm_si − 1) slave particles. The number of subswarms sn decreases linearly with the iteration of the algorithm; at the iter generation, it can be calculated by Equation (3).

$$s{n}^{(iter)}=ceil\left({\displaystyle \frac{M}{2}}(1-{\displaystyle \frac{iter}{MaxGen+1}})\right)$$

(3)

where MaxGen represents the maximum number of iterations of the algorithm. Figure 2 clearly explains the proposed multiswarm segmentation scheme with a master–slave structure, which is divided into three steps.

First, the superior particles are identified. The individuals are sorted according to the fitness value of the particles, and the top of the M/2 particles are selected as the superior particle set $\mathit{ST}=\{\mathit{s}{\mathit{t}}_1,\mathit{s}{\mathit{t}}_2,\dots ,\mathit{s}{\mathit{t}}_g\},g=M/2$, from which the master particles will be selected.

Second, the master particles are selected. The distribution of the subswarms is closely related to the leader particle, and a more balanced distribution is conducive to the particle exploring a wider area. Therefore, to ensure a more balanced distribution of each subswarm, we took the Euclidian distance between particles as a measure and selected the combination that contained sn particles with the largest sum of distances in the superior particle set ST as the set of master particles, $\mathit{Maste}{\mathit{r}}_i={(\mathit{mas}{\mathit{t}}_1,\mathit{mas}{\mathit{t}}_2,\dots ,\mathit{mas}{\mathit{t}}_{sn})}^T$,$\mathit{Master}\subseteq \mathit{ST}$. The distance sum of possible combinations, $\mathit{Cs}=\{\mathit{c}{\mathit{s}}_1,\dots ,\mathit{c}{\mathit{s}}_{sn}\}$ and $\mathit{cs}\subseteq \mathit{ST}$ can be calculated by Equations (4) and (5).

$$\max Sum\_d={\displaystyle \sum _{i=1}^{sn-1}{\displaystyle \sum _{j=i+1}^{sn}dis{t}_{i,j}}}$$

(4)

$$dis{t}_{i,j}=\sqrt{{\displaystyle \sum _{k=1}^D{(c{s}_i^k-c{s}_j^k)}^2}}, (i\ne j)$$

(5)

where dist_i,j denotes the distance between two different D-dimensional individuals, cs_i and cs_j, and Sum_d represents the distance sum between all individuals in the Cs. Then, the combination of Cs with the largest Sum_d can be selected as the Master by traversing all possible combinations.

Finally, the slave particles are assigned. An individual master particle, mast_si, in Master leads to a subswarm, subpop_si, and all other individuals, except those who have been selected as leader particles, should be assigned to subswarms. When assigning slave particles, the master particle is taken as the center of the subswarm, and the particle closest to itself is selected as its slave particle. Each subswarm competes for its own slave particles until the number of its slave particles reaches the upper limit. In detail, a particle x_i is randomly selected from all of the particles except the master particle. If x_i has the smallest Euclidean distance to mast_k, it will be assigned to subpop_k. Different particle numbers are allowed in each subswarm.

(2)

The multiswarm dynamic adjustment strategy

In the process of evolution, due to the small scale, the global exploration ability of the subswarm is limited, which makes it easy to lose diversity and fall into the local optimal region. To solve this problem, a multiswarm dynamic adjustment strategy was designed in which the evolutionary ability of each subswarm is evaluated to determine when to update and restructure all the subswarms. When the evolutionary ability of the subswarm is weakened and its advantage cannot be effectively brought into play, the population is reorganized so that the subswarms can exchange information and maintain good diversity. In detail, the evolutionary ability of the population is measured for each interval iteration parameter G. If the subswarm finds a better position after generation G, it means that it still has evolutionary abilities. In contrast, this means that the subswarm has lost its evolution abilities. Finally, the average evolutionary ability of the subswarm (SPEA_ave) is used as a measure of the evolutionary ability of the population and can be calculated by Equations (6) and (7).

$$SPE{A}_k=\{\begin{array}{l}\begin{array}{cc}1& Lbestva{l}_k^{iter+G}<Lbestva{l}_k^{iter}\end{array}\\ \begin{array}{cc}0& Lbestva{l}_k^{iter+G}\ge Lbestva{l}_k^{iter}\end{array}\end{array}$$

(6)

$$SPE{A}_{ave}={\displaystyle \frac{1}{sn}}{\displaystyle \sum _{k=1}^{sn}SPE{A}_k}$$

(7)

where SPEA_k is the evolutionary ability of the k-th subswarm, $k=1,2,\dots ,sn$. $Lbestva{l}_k^{iter}$ and $Lbestva{l}_k^{iter+G}$ represent the fitness values of the k-th subswarm at the local neighbors’ best positions in the iter and iter + G generation, respectively. At the same time, a recombination parameter, Rc, is defined. When condition $SPE{A}_{ave}<Rc$ is satisfied, the subswarm will be redivided. The values of the interval iteration parameter, G, and recombination parameter, Rc, are investigated in Section 4.1. In summary, the detailed process of the MDMS strategy is shown in Algorithm 1.

Algorithm 1. MDMS_Phase
1:	Calculate sn using Equation (3);
2:	[~,index_f] = sort(fitness);
3:	Index of superior particles: ST_temp = index_f (1:M/2);
4:	Index of all potential combinations of master particles Comb = nchoosek(ST_temp, sn);
5:	Calculate all combinations of distance sum Sum_d(Pos(Comb)) using Equations (4) and (5);
6:	Index of the combination Cs: [~, index_m] = max (Sum_d);
7:	The index of master particles Master = Comb(index_m,:);
8:	for k = 1: sn
9:	subx(1,:) = Pos(Master(k),:), subp(1,:) = Pbest(Master(k),:);
10:	end for
11:	for l = 1:M
12:	if ~ismember(l,Master)
13:	ms_d = pdist2(Pos(l), Pos(Master,:));
14:	[~,index_s] = min(ms_d);
15:	subx{index_s}(end + 1,:) = Pos(l,:), subp{index_s}(end + 1,:) = Pbest(l,:);
16:	end if
17:	end for

3.2. A Multimodel Learning Strategy with Reward and Punishment Mechanisms

In the evolution process of the PSO algorithm, particles find better positions by constantly learning from their own historical experience and the historical experience of their neighbors. The learning characteristics are mainly reflected in the update of particle velocity v. A good learning example should be able to adapt to the different needs of the algorithm evolution process; that is, in the initial stage, it should be able to maintain better diversity to explore a wider area in the search space, and in the final stage, it should make the algorithm have better local search abilities to improve the solution accuracy of the algorithm and save the number of evaluations. However, it is difficult for a single learning strategy to satisfy all the needs of the algorithm in the different stages. Based on this, this paper used the reward and punishment mechanism to manage three learning strategies, the MCL, LDL, and UL strategies, so that the different subswarms can dynamically choose learning strategies conducive to their own evolution.

(1) 

MCL strategy

The CL strategy can obtain more abundant information from the entire population in the process of algorithm evolution to help particles evolve and keep the population exploring a wider area in the search space. In the multiswarm PSO, different subswarms may contain different information elements. Therefore, sufficient information sharing among the different subswarms is conducive to the evolution of the population. Based on this, this paper adjusted the CL strategy and proposed an MCL strategy. When selecting the learning template, individuals from different subswarms are selected as learning objects to promote the information interaction between subswarms in the process of evolution. The update of speed v is shown in Equations (8) and (9).

$$v_{i,j}^{(t+1)}=w{v}_{i,j}^{(t)}+c_1{r}_{1,j}(MCLPbes{t}_{i,j}^{(t)}-x_{i,j}^{(t)})$$

(8)

$$MCLPbes{t}_{i,j}^{(t)}=\{\begin{array}{l}Pbes{t}_{i,j}^{(t)} (rand>P{c}_i)\\ Mpbes{t}_{i,j}^{(t)} (rand\le P{c}_i)\end{array}$$

(9)

where Mpbest_i is the particle x_i of the learning example. The particle from each subswarm is randomly selected to form a learning set, in which Mpbest_i,j is consistent with the dimension corresponding to the pbest of the winner in the learning set. Different particles adopt different learning probabilities, Pc, which are related to the number of particles in the subswarms where the particle is located sm.

$$P{c}_i=0.05+{\displaystyle \frac{0.45(e^{10(i-1)/(s{m}_k-1)}-1)}{e^{10}-1}}$$

(10)

(2) 

LDL strategy

In the process of population evolution, the effective local exploitation of subswarms can better improve the overall performance of the algorithm. However, making full use of the local optimal information can speed up the development of the local areas. Using the idea of dimension learning for reference, this paper constructed an update template through the historical optimal dimension information of the particles and the global optimal dimension information of the population to update the particle speed and position, as shown in Equations (11) and (12):

$$v_{i,j}^{(t+1)}=w{v}_{i,j}^{(t)}+c_1{r}_{1,j}(LDPbes{t}_{i,j}^{(t)}-x_{i,j}^{(t)})$$

(11)

$$LDPbes{t}_{i,j}^{(t)}=\{\begin{array}{l}Pbes{t}_{i,j} (rand>P{l}_i)\\ Lbes{t}_{k,j} (rand\le P{l}_i)\end{array}$$

(12)

The selection probability Pl is used to manage the probability that the dimension information of the x_i historical optimal position Pbest_i and its subswarm historical optimal position Lbest_k are selected. In this paper, when updating the particle speed, we put Pbest_i and Lbest_k in the same important position, so we assigned Pl = 0.5. In detail, for each dimension of x_i, if the generated random number rand is greater than the selection probability Pl, the information of the corresponding dimension in LDPbest_i is selected from Pbest_i; otherwise, the Lbest_k where particle x_i belongs is selected.

(3) 

UL strategy

The MCL strategy can extract more individual information, maintain better diversity of the population, and promote particle exploration but it also limits the exploitation ability. The LDL strategy strengthens the particle exploitation ability through the information of the particle’s pbest and Lbest of the subpopulation where the particle belongs, however, it is also easier to fall into a local optimization. To balance the advantages and disadvantages of the MCL strategy and LDL strategy, the UL strategy was proposed. Randomly select $H(H\le sn)$ subswarms, and use the Lbest of the selected subswarms to construct a unit learning example Ubest, that is, the center of the historical optimal position of all the selected subswarms is used to guide the evolution of the target particles. The particle’s speed update is affected by the combination of the particle’s pbest and Ubest, the best location in the history of the individual, and the update formulas are Equations (13) and (14):

$$v_{i,j}^{(t+1)}=w{v}_{i,j}^{(t)}+c_1{r}_{1,j}(Pbes{t}_{i,j}^{(t)}-x_{i,j}^{(t)})+c_2{r}_{2,j}(Ubes{t}_j^{(t)}-x_{i,j}^{(t)})$$

(13)

$$Ubest={\displaystyle \frac{1}{H}}{\displaystyle \sum _{h=1}^{H}Lbes{t}_h}$$

(14)

At different evolutionary stages, learning examples have different effects on the population evolution. For example, at the initial stage of the evolution, selecting the MCL learning model can meet the needs of the algorithm to fully explore the search space at the initial stage. At the end of evolution, the LDL strategy was adopted to accelerate the convergence of the algorithm. Therefore, to meet the needs of different evolutionary stages of the algorithm and give better play to the advantages of the multiple learning templates, this paper proposed a learning model selection mechanism with reward and punishment mechanisms. By selecting the benchmark function and the reward and punishment factors, the benchmark function ensures the overall selection probability change trend of the different templates in the evolution, and the reward and punishment factors fine tune the selected probability of the templates according to the evolutionary stage. There is more effective control of the probability that different learning templates are selected in the iteration to affect the evolution process of the population to improve the performance of the algorithm. Therefore, we built two strategy selection functions, P_ls1 and P_ls2, to manage three different learning strategies, which can be calculated by Equation (15). As shown in Figure 3, P_ls1 is used to divide the selection probability of the LDL and MCL strategies, and P_ls2 is the selection probability of the MCL and UL strategies. $Rand (Rand\in [0,1])$ is a random parameter that conforms to a uniform distribution. If $Rand<P_{ls1}$, the subswarm selects the LDL strategy to update the individual. If $P_{ls1}\le Rand<P_{ls2}$, the subswarm selects the MCL strategy to update the individual. If $Rand\ge P_{ls2}$, the subswarm selects the UL strategy to update the individual. MaxFEs is the maximum number of fitness evaluations.

$$\{\begin{array}{l}{P}_{ls1}=0.1+0.4\times {10}^{4 \times ({\displaystyle \frac{fitcount}{MaxFEs}}-1)}+r{p}_1\\ P_{ls2}=0.66-r{p}_2\end{array}$$

(15)

where rp₁ is the reward and punishment factor of the LDL strategy relative to the MCL strategy, and rp₂ is the reward and punishment factor of the UL strategy relative to the MCL strategy, which was used to better balance the strategy selection at the different evolutionary stages. The reward and punishment factor was constructed based on the promotion abilities of the learning strategies in different stages for the individual evolution, that is, the improvement rate (Ir) of the fitness. In this paper, considering that the objective f is the minimum, that is, the smaller the value f(x), the better the performance of x. Therefore, the Ir of the learning strategy on the selected particles in generation t is defined as Equation (16).

$$Ir =\{\begin{array}{l}0 (ln=0)\\ {\displaystyle \frac{1}{ln}}{\displaystyle \sum _{i=1}^{ln}\frac{\mathit{f}\left({\mathit{x}}_i^{t-1}\right)-\mathit{f}\left({\mathit{x}}_i^t\right)}{e^{\Vert {\mathit{x}}_i^{t-1}-{\mathit{x}}_i^t\Vert }}} (ln>0)\end{array}$$

(16)

where ln is the number of particles that choose a learning strategy, $\Vert {\mathit{x}}_i^{t-1}-{\mathit{x}}_i^t\Vert$ indicates the Euclidean distance between ${\mathit{x}}_i^{t-1}$ and ${\mathit{x}}_i^t$, and $\mathit{f}\left({\mathit{x}}_i^t\right)$ is the fitness of ${\mathit{x}}_i^t$. Therefore, the improvement rates of the three learning strategies LDL, MCL, and UL are Ir_LD, Ir_CL, and Ir_UL, respectively.

It should be noted that in an independent iteration, the influence rate of each learning strategy on the individual evolution is random. Therefore, rp₁ and rp₂ cannot be directly used to observe the promotion effect of the learning model on the individual evolution at this stage. In this article, we used the technique to smooth it to reduce the influence of randomness in a single generation and obtained the performance change trend of the different learning templates at the different evolution stages. The reward and punishment factors rp₁ and rp₂ for learning strategy selection can be calculated by Equation (17).

$$\{\begin{array}{l}r{p}_1=\mathit{smooth}\left(0.5-{\displaystyle \frac{1}{1+e^{\frac{I{r}_{LD}-I{r}_{CL}}{I{r}_{CL}+I{r}_{LD}+\sigma }}}}\right)\\ r{p}_2=\mathit{smooth}\left(0.5-{\displaystyle \frac{1}{1+e^{\frac{I{r}_{UL}-I{r}_{CL}}{I{r}_{CL}+I{r}_{UL}+\sigma }}}}\right)\end{array}$$

(17)

where $\mathit{smooth}(\cdot )$ is a Savitzky–Golay smooth function and $\sigma$ is a minimum value, which ensures that the denominator is not equal to 0.

The change in the strategy selection function with the reward and punishment factors on benchmark function f1 is shown in Figure 4. The MCL strategy has a better ability to promote the evolution of the individuals in the middle of evolution. Therefore, at this stage, the probability of the strategy being selected is increased.

In MLDMS-PSO, the three learning strategies are properly implemented to enable the different strategies to cooperate with each other, which is conducive to mining potential promising information to help the particles evolve. For the different evolution stages, the strategy selection function is used to effectively control the selection of advantageous particle update strategies, which balances exploration and exploitation and improves the overall performance. The process of MLDMS-PSO is shown in Algorithm 2.

Algorithm 2. MLDMS-PSO_Phase
1:	Set parameter M,G,Rc;SPEAave = 0;delta_t = G;
2:	Initialize the position and velocity of all particles x and v, respectively;
3:	Evaluate x: fitness = fit(x);
4:	Pbest = x, Gbest = [Pbesti | min(fit(Pbesti)];
5:	MDMS_Phase;
6:	Lbestk = [subp(i,:) | min(fit(subp{k}(i,:)))];
7:	While (iter ≤ MaxFEs)
8:	if delta_t ≥ G
9:	Calculate average evolutionary ability SPEAave using Equations (5) and (6);
10:	if SPEAave < Rc
11:	MDMS_Phase;
12:	end if
13:	end if
14:	for k = 1,2, …, sn do
15:	Calculate Pls1 and Pls2 using Equations (15)–(17);
16:	if a < Pls1
17:	Update particles of subpop{k} using Equations (11), (12) and (2);
18:	end if
19:	if Pls1 ≤ a < Pls2
20:	Update particles of subpop{k} using Equations (8), (9) and (2);
21:	end if
22:	if Pls2 ≤ a
23:	Update particles of subpop{k} using Equations (13), (14) and (2);
24:	end if
25:	Evaluate particles of subpop{k} fitness
26:	Update Lbestk;
27:	end for
28:	Update Gbest;
29:	end while

4. Experimental Studies

To calibrate the parameters and test the performance of the proposed algorithm, we conducted a series of experiments based on 29 benchmark functions from the CEC2017 test suite. These benchmark functions were divided into four different categories, that is, two unimodal functions (f1 and f3. f2 was removed because it is unstable on issues of higher dimensions), seven multimodal functions (f4–f10), ten hybrid functions (f11–f20), and ten composition functions (f21–f30), which are shown in

Table 1. The information of the benchmark function used in this paper.

No.	Function	Range	Category	fopt
f1	Shifted and Rotated Bent Cigar Function	[−100, 100]	UN	100
f3	Shifted and Rotated Zakharov Function	[−100, 100]	UN	300
f4	Shifted and Rotated Rosenbrock’s Function	[−100, 100]	MN	400
f5	Shifted and Rotated Rastrigin’s Function	[−100, 100]	MN	500
f6	Shifted and Rotated Expanded Scaffer’s F6 Function	[−100, 100]	MN	600
f7	Shifted and Rotated Lunacek Bi-Rastrigin Function	[−100, 100]	MN	700
f8	Shifted and Rotated Non-Continuous Rastrigin’s Function	[−100, 100]	MN	800
f9	Shifted and Rotated Levy Function	[−100, 100]	MN	900
f10	Shifted and Rotated Schwefel’s Function	[−100, 100]	MN	1000
f11	Hybrid Function 1 (N = 3)	[−100, 100]	H	1100
f12	Hybrid Function 2 (N = 3)	[−100, 100]	H	1200
f13	Hybrid Function 3 (N = 3)	[−100, 100]	H	1300
f14	Hybrid Function 4 (N = 4)	[−100, 100]	H	1400
f15	Hybrid Function 5 (N = 4)	[−100, 100]	H	1500
f16	Hybrid Function 6 (N = 4)	[−100, 100]	H	1600
f17	Hybrid Function 6 (N = 5)	[−100, 100]	H	1700
f18	Hybrid Function 6 (N = 5)	[−100, 100]	H	1800
f19	Hybrid Function 6 (N = 5)	[−100, 100]	H	1900
f20	Hybrid Function 6 (N = 6)	[−100, 100]	H	2000
f21	Composition Function 1 (N = 3)	[−100, 100]	C	2100
f22	Composition Function 2 (N = 3)	[−100, 100]	C	2200
f23	Composition Function 3 (N = 4)	[−100, 100]	C	2300
f24	Composition Function 4 (N = 4)	[−100, 100]	C	2400
f25	Composition Function 5 (N = 5)	[−100, 100]	C	2500
f26	Composition Function 6 (N = 5)	[−100, 100]	C	2600
f27	Composition Function 7 (N = 6)	[−100, 100]	C	2700
f28	Composition Function 8 (N = 6)	[−100, 100]	C	2800
f29	Composition Function 9 (N = 3)	[−100, 100]	C	2900
f30	Composition Function 10 (N = 3)	[−100, 100]	C	3000

. All the experiments were implemented in MATLAB R2021a on a computer with an Intel Core i7-10700 CPU @2.90 GHz (Intel Corporation, Santa Clara, CA, USA) and 32 GB RAM running Windows 10x64 Edition.

4.1. Parameter Tuning

The value of some parameters may greatly affect the performance of the algorithm. In the proposed MLDMSPSO, two important parameters, the interval iteration parameter G and the recombination parameter Rc, need to be calibrated. The parameter tuning used the same population size of 40, maximum number of fitness evaluations (MaxFEs) of 1.00 × 10⁵, run time of 51, and 10 dimensions.

A smaller value for parameter G and a larger value for Rc may lead to frequent reorganization of the subswarms in the iterative process, which makes the subpopulation unable to make full use of the internal information to improve itself and fully exploit the local area where it is located. In contrast, for a larger parameter G and a smaller Rc, the population may have the risk of a stagnant evolution and a waste of the evaluation times. Therefore, to obtain an insightful view of how the interval iteration parameter G and recombination parameter Rc affect the performance of MLDMS-PSO, this paper tested 48 different combinations of G values, from 5 to 20, when Rc was 0.1, 0.2, and 0.3. The experimental results were ranked by the average rank with the mean of 29 benchmark functions as the measurement standards. When the average rank value corresponding to a combination of G and Rc is smaller, it indicates that the combination performs better in terms of overall performance and can exhibit better performance on more functions; on in contrast, the larger the average rank value, the weaker its comprehensiveness.

The average rank value of the 29 benchmark functions was counted when MLDMS-PSO, using different G and Rc, plotted these, as shown in Figure 5. It can be clearly seen that the algorithm exhibited a similar performance at different Rc values. The average rank value first decreased and then increased with the increase in G, indicating that larger or smaller G values could not help the MLDMS-PSO achieve better performance; on the other hand, when the value of G was close to the middle region of the value range, it was beneficial for MLDMS-PSO to obtain a better ranking on more functions and had better overall performance. When the iteration parameter G = 12 and recombination parameter Rc = 0.1, the average rank was 19.34, which was lower than all of the other combinations of G and Rc for all test functions and could bring the best performance to MLDMS-PSO. Therefore, the iteration parameter G = 12 and recombination parameter Rc = 0.1 were selected for a further performance evaluation of the proposed MLDMS-PSO.

4.2. Comparison with Other PSO Variants

To compare and analyze the performance of the proposed MLDMS-PSO, nine other state-of-the-art PSO variations were selected and divided into three types.

There are four PSO variants with different learning strategies, as follows:

➢

Traditional global PSO algorithms with inertia weight (GPSO) [17].

➢

Full information PSO (FIPS) [47]. The FIPS implements the ring topology.

➢

Comprehensive learning PSO (CLPSO) [21]. All particles update the dimension information of their velocity using pbest and learn from the pbest of different particles.

➢

Competitive and cooperative (CC) PSO with ISM (CCPSO-ISM) [37]. The CC operator based on the ISM is designed to use the shared information properly and efficiently.

Two PSOs adopt multiple learning strategies, as follows:

➢

A PSO with the multi-exemplar and forgetting ability (XPSO) [48]. XPSO uses two exemplars to update the velocity and configures distinct forgetting abilities to different particles.

➢

Two-swarm learning PSO (TSLPSO) [20]. In TSLPSO, the dimensional learning strategy and CL strategy are used to guide the local and global search of the particles, respectively.

Three PSOs divide the whole population into multiple swarms.

➢

Dynamic multiple-swarm PSO (DMS-PSO) [11]. DMS-PSO adopts the multiple-swarm strategy and these swarms can be regrouped frequently.

➢

Multiswarm PSO with a dynamic learning strategy (PSO-DLS) [16]. In PSO-DLS, the particle classification mechanism and the dynamic control mechanism of the strategy promote an information exchange among the subswarms.

➢

Heterogeneous comprehensive learning and dynamic multiswarm PSO with two mutation operators (HCLDMS-PSO) [49]. In HCLDMS-PSO, a CL strategy and a modified DMS strategy are used to construct the exploitation subpopulation exemplar and the exploration subpopulation exemplar, respectively.

The parameter settings of the PSO variants above-mentioned and the proposed MLDMS-PSO are listed in

Table 2. Parameter settings of the ten PSOs.

Algorithm	Parameters Setting	Year
GPSO	$w=0.9\sim 0.4$, $c_1=c_2=2$	1998
FIPS	$\chi =0.729$, ${\displaystyle \sum c_i=4.1}$	2004
CLPSO	$w=0.9\sim 0.4$, $c=1.49445$	2006
CCPSO-ISM	P = 0.05, G = 5, w = 0.6, c = 2	2015
TSLPSO	$w=0.9\sim 0.4$, $c_1=c_2=1.49445$, $c_3=0.5\sim 2.5$	2019
XPSO	$\eta =0.2$, $Sta{g}_{\max }=5$, $p=0.5$	2020
DMS-PSO	$w=0.9\sim 0.2$, $c_1=c_2=1.49445$	2005
PSO-DLS	$w=0.9\sim 0.4$, $c_1=c_2=1.49445$	2017
HCLDMS-PSO	$w=0.99\sim 0.29$, $c_1=2.5\sim 0.5$, $c_2=0.5\sim 2.5$, $P_m=0.1$	2020
MLDMS-PSO	$w=0.9\sim 0.4$, $c_1=c_2=1.49445$, G = 12, Rc = 0.1	-

. To make a fair comparison between the proposed MLDMS-PSO and the other state-of-the-art PSO algorithms, the mean error (mean), standard deviation (std), and rank mean value (rank) of 10 PSOs solution were calculated as a basic performance measure to evaluate the accuracy of the algorithm. The experiments were conducted on dimensions of D = 10, 30, and 50. Each algorithm was run 51 times independently on 10-D, 30-D, and 50-D problems for the 29 benchmark functions, and the termination condition was set to MaxFEs = 10,000 × D. The results of 10-D, 30-D, and 50-D are shown in Appendix A, Appendix B, and Appendix C, respectively, where the best experimental result among all of the test functions is shown in bold. Based on this result, all algorithms were sorted according to the average rank (Ave rank) of the 29 functions as the final rank (Final rank). The Ave rank is the average value of the algorithm’s experimental results ranking on each benchmark function. The Final rank is the result of sorting all algorithms by Ave rank. To compare the comprehensive performance of MLDMS-PSO and nine other state-of-the-art PSO variations in different dimensions, as shown in

Table 3. The comprehensive rank of MLDMS-PSO and other PSO variations for 10-D, 30-D, and 50-D problems.

Dim	Index	GPSO	FIPS	CLPSO	CCPSO-ISM	XPSO	TSLPSO	DMS-PSO	PSO-DLS	HCLDMS-PSO	MLDMS-PSO
10-D	Ave rank	8.62	9.93	4.28	4.14	5.69	6.72	3.07	5.12	4.41	2.41
30-D	Ave rank	8.97	10	5.03	4.72	6.52	4.55	4.93	3.97	3.28	3
50-D	Ave rank	8.97	10	4.9	5.1	5.9	5.79	4.86	3.41	3.07	3
CMean rank		8.85	9.98	4.74	4.65	6.04	5.69	4.29	4.17	3.59	2.80
Crank		9	10	6	5	8	7	4	3	2	1

, we calculated the mean of each algorithm’s Ave rank in different dimensions as the comprehensive mean rank (CMean rank). Based on this CMean rank, the comprehensive ranking (Crank) of each algorithm was obtained.

4.3. Results Analysis and Discussion

4.3.1. Solutions Accuracy

The comparison of the experimental results for the 10-D problems is shown in Appendix A. The proposed MLDMS-PSO achieved the best results on f6, f9, f11, f13, f14, f16, f18, f20, f23, and f29. The second-best solution was obtained on f12, f27, and f30. The result ranked third on f1, f4, f5, f8, f15, f17, f19, f21, f22, f24, f26, and f28, and the result ranked fourth on f7 and f10, which was better than most comparison algorithms. On f3, although the mean value of its solution ranked lower, the accuracy of the solution was satisfactory, which was 6.96 × 10⁻¹². In addition, MLDMS-PSO achieved the minimum average rank. Therefore, MLDMS-PSO outperformed the other comparison algorithms on 10-D problems.

The comparison of the experimental results for the 30-D problems is shown in Appendix B. TSLPSO and XPSO had the best performance on unimodal functions f1 and f3, respectively. The proposed MLDMS-PSO ranked third and fifth on f1 and f3, respectively. In the simple multimodal functions, hybrid function, and composition function, the proposed MLDMS-PSO achieved the best solution on f6, f9, f11, f12, f17, f18, f20, f22, f28, and f29 and the second-best solution on f16, f25, and f30. CCPSO-ISM obtained the best performance on f6, f13, f15, f19, and f26. PSO-DLS obtained the best solution on f5, f7, f8, f21, f23, and f24. DMS-PSO found the smallest mean value of all solutions on f4, f14, f16, f25, f27, and f30. HCLDMS-PSO showed the best results on f10. In total, depending on the final rank, the MLDMS-PSO algorithm was the first. Hence, MLDMS-PSO consistently performed well and achieved the best overall performance on the 30-D problems.

As shown in Appendix C, for the 50-D problems, it can be seen that the proposed MLDMS-PSO had the highest performance on the whole test suite. Compared with other comparison algorithms, it achieved the best results on f6, f11, f13, f14, f16, f17, f20, and f29. Depending on the final rank, the MLDMS-PSO achieved the optimal performance, ranking first on eight problems, second on two problems, and third on eleven problems out of the twenty-nine benchmark problems. In addition, it is worth noting that the average ranking of the four algorithms with dynamic multiswarm DMS-PSO, PSO-DLS, HCLDMS-PSO, and MLDMS-PSO, was to a large extent, ahead of the other comparison algorithms, which also proves that a dynamic multiswarm algorithm can better implement a full exploration of the search space, effectively avoiding the population from falling into a local optimization, which can play a positive role in solving larger-scale problems.

Table 3 clearly shows that the CMean rank value of the MLDMS-PSO was the smallest among all of the compared algorithms, at 2.80. This result placed the MLDMS-PSO at the top of Crank, which is consistent with our previous discussion and further confirms the advantages of the MLDMS-PSO.

4.3.2. Convergence Process Analysis

The convergence speed is also an important indicator of the performance of evolutionary algorithms. In this part, we took the experimental results of the 30-D problems as an example to analyze the differences in the convergence process of the 10 algorithms. The results of the convergence process were classified according to the type of benchmark functions in the test set. Due to space limitations, the convergence of some functions was reduced, as shown in Figure 6, Figure 7, Figure 8 and Figure 9. The shaded area in the figures has been enlarged to more clearly illustrate the convergence process of the algorithms. It should be noted that in this experiment, the convergence curve showed the average results of 51 independent runs.

For the unimodal functions, the convergence process of the algorithms is shown in Figure 6. TSLPSO and XPSO achieved faster convergence speeds and superior performances on f1 and f3, respectively. However, MLDMS-PSO was limited by the population dynamic reorganization scheme, which weakened its exploitation ability and made it show more stable results on two single-mode functions.

For the multimodal functions, the convergence process of the algorithms is shown in Figure 7. The classic PSO algorithm had a sharp convergence on f5, f7, and f8 but its performance was poor. MLDMS-PSO, HCLDMS-PSO, and PSO-DLS had similar features in f5, f6, f7, f8, and f9, that is, they could accelerate the convergence when approaching the late evolutionary stage, which shows that the dynamic multiswarm strategy has a stronger ability to jump out of the local optimum.

For the hybrid functions, the convergence process of the algorithms is shown in Figure 8. MLDMS-PSO achieved the best performance in f14, f16, and f17. In most hybrid functions, the convergence process was slow. The reason for this is that the effective management of multiple learning strategies can continuously obtain more favorable information, making the population continue to evolve. MLDMS-PSO also demonstrated good exploratory ability in f16 and f20 during the later stages of evolution.

For the composition functions, the convergence process of the algorithms is shown in Figure 9. MLDMS-PSO obtained competitive results in f22, f23, f27, f29, and f30, but its advantages were not obvious. By further improving the algorithm, we could make it more accurate in solving the composite function and improving the practicability of the algorithm, which is a direction that needs to be constantly explored.

In summary, MLDMS-PSO adopts a multiswarm architecture, allowing various subswarms to fully explore the solution space and demonstrate strong exploration ability in multimodal, hybrid, and composition functions. However, the cost of powerful global search capability is the loss of some local development ability, which slows down its local development speed. Compared to the improvement in solution accuracy, the slight loss in convergence speed can be accepted.

4.3.3. Statistical Results of Solutions

(1) Wilcoxon rank sum test: A Wilcoxon rank sum test [50] is a nonparametric sample test that is based on the rank order of samples and not the average value, which is used to compare the difference between the distribution positions of two populations to which two independent samples belong. In this paper, the nonparametric Wilcoxon rank sum test was used to measure the statistical significance difference between the proposed MLDMS-PSO and the other comparison algorithms. In this experiment, the distribution of the results obtained by the MLDMS-PSO algorithm and the other PSO variants on all the benchmark functions through 51 simulation runs on the 10-D, 30-D, and 50-D problems were compared, in which its significance level was 0.05. The statistical results are recorded in

Table 4. Statistical results of the Wilcoxon rank sum test for 10-D, 30-D, and 50-D problems.

Algorithm	Dim	N+	N=	N−	CP	Algorithm	Dim	N+	N=	N−	CP
MLDMS-PSO vs. GPSO	10-D	28	0	1	27	MLDMS-PSO vs. TSLPSO	10-D	11	14	4	7
	30-D	28	1	0	28		30-D	17	4	8	9
	50-D	27	2	0	27		50-D	18	5	6	12
MLDMS-PSO vs. FIPS	10-D	29	0	0	29	MLDMS-PSO vs. DMS-PSO	10-D	22	3	4	18
	30-D	29	0	0	29		30-D	20	4	5	15
	50-D	29	0	0	29		50-D	16	8	5	11
MLDMS-PSO vs. CLPSO	10-D	15	9	5	10	MLDMS-PSO vs. PSO-DLS	10-D	22	3	4	18
	30-D	19	7	3	16		30-D	15	2	12	3
	50-D	18	8	3	15		50-D	15	5	9	6
MLDMS-PSO vs. CCPSO-ISM	10-D	18	3	8	10	MLDMS-PSO vs. HCLDMS-PSO	10-D	21	6	2	19
	30-D	19	1	9	10		30-D	12	8	9	3
	50-D	18	6	5	13		50-D	12	7	10	2
MLDMS-PSO vs. XPSO	10-D	26	1	2	24
	30-D	23	4	2	21
	50-D	20	7	2	18

. The symbols “N+”, “N−”, and “N=” indicate the number that MLDMS-PSO was significantly better than, significantly worse than, and almost the same as the corresponding competitor algorithm, respectively. The comprehensive performance (CP) was equal to the sum of “N+” minus the sum of “N−”.

Table 4 shows that MLDMS-PSO was significantly better than the other PSO variants on most 10-D, 30-D, and 50-D problems. This view is also supported by the fact that all values of “N+” were greater than “N−”. In addition, the CP of all the comparison algorithms was positive, which also showed that the new proposed MLDMS-PSO had the best comprehensive performance. HCLDMS-PSO performed well on the 30-D and 50-D test functions with CP values of 3 and 2, respectively, and its performance was close to that of MLDMS-PSO. However, in the 10-D problems, its performance was significantly weaker than MLDMS-PSO.

(2) Friedman Test: In this paper, the Friedman test of the mean was used to compare the overall performance of MLDMS-PSO and the other comparison algorithms on different benchmark functions. The Friedman test results of the 10-D, 30-D, and 50-D problems are given in

Table 5. Statistical rank of the Friedman test.

Overall Rank	Comprehensive Rank		10-D		30-D		50-D
	Algorithm	Rank	Algorithm	Rank	Algorithm	Rank	Algorithm	Rank
1	MLDMS-PSO	2.862	MLDMS-PSO	2.483	MLDMS-PSO	3.003	MLDMS-PSO	3.000
2	HCLDMS-PSO	3.598	TSLPSO	3.138	HCLDMS-PSO	3.281	HCLDMS-PSO	3.069
3	PSO-DLS	4.379	CCPSO-ISM	4.207	PSO-DLS	3.966	PSO-DLS	3.414
4	TSLPSO	4.494	CLPSO	4.328	TSLPSO	4.552	DMS-PSO	4.862
5	CCPSO-ISM	4.678	HCLDMS-PSO	4.483	CCPSO-ISM	4.724	CLPSO	4.897
6	CLPSO	4.753	DMS-PSO	5.121	DMS-PSO	4.897	CCPSO-ISM	5.103
7	DMS-PSO	4.960	PSO-DLS	5.759	CLPSO	5.034	TSLPSO	5.793
8	XPSO	6.402	XPSO	6.793	XPSO	6.517	XPSO	5.897
9	GPSO	8.874	GPSO	8.690	GPSO	8.966	GPSO	8.966
10	FIPS	10.000	FIPS	10.000	FIPS	10.000	FIPS	10.000
			p-value	3.23 × 10−30	p-value	1.53 × 10−28	p-value	1.93 × 10−29

, listing each algorithm and its ranking (ascending order). Finally, a comprehensive ranking based on the mean of the results was used to measure the overall performance of all of the algorithms.

From Table 5, we can see that the proposed MLDMS-PSO algorithm in this paper ranked first in the 10-D, 30-D, and 50-D problems and obtained a better overall performance than the other comparison algorithms, which was the same as the results obtained by the Wilcoxon rank sum test. In detail, TSLPSO and CCPSO-ISM ranked behind MLDMS-PSO in the 10-D problems, and HCLDMS-PSO and PSO-DLS ranked second and third in the 30- and 50-dimensional problems, respectively. This is because the HCLDMS-PSO and PSO-DLS could obtain better population diversity during the evolution than TSLPSO and CCPSO-ISM, strengthening the exploration ability of the algorithm but also weakening the exploitation ability of the algorithm to a certain extent. The MLDMS-PSO made the algorithm more compatible with exploration and exploitation through population contraction and an adaptive learning strategy adjustment, so it had a better performance on the 10-D, 30-D, and 50-D problems.

5. Conclusions

A multipopulation metaheuristic algorithm can effectively alleviate the loss of population diversity and the risk of falling into the local optima, especially for multimode optimization problems. Based on this and the distance measurement, the MLDMS-PSO algorithm adopted a master–slave structure to divide the entire population into multiple subswarms, and through linear population reduction, it could better meet the needs of the population diversity in the different evolutionary stages. Inspired by this phenomenon, the MLDMS-PSO algorithm introduces three learning strategies, the LDL strategy, the MCL strategy and the UL strategy, to guide particle updating. At the same time, two probability selection functions are used to manage the three learning strategies, and the reward and punishment factor constructed based on the change of fitness (i.e., the improvement rate) is regarded as an additional standard. In the different stages, the probability of the selected learning strategy can be dynamically adjusted according to the change in fitness so that particles can choose more advantageous evolution strategies for different fitness terrains and improve the efficiency of the evolution. In addition, the choice of multiple learning models can better balance exploration and development. To verify the effectiveness of the MLDMS-PSO, the CEC2017 test suite was used for the performance comparisons. The results show that compared with the other existing advanced particle swarm optimization algorithms, the proposed MLDMS-PSO algorithm had better performance in multimodal, hybrid, and composition problems. During the evolution process, it could better maintain population diversity, drive continuous population evolution, strengthen the algorithm’s exploration ability, and obtain more accurate solutions. To simplify the algorithm, a linear reduction was adopted to control the number of subswarms without considering the impact of population number on the evolutionary efficiency at different stages of evolution. This sacrificed its exploitation ability to some extent, slowed down the convergence speed, and also made its performance on single-mode problems less accurate, which are not ideal.

A good population diversity can enhance the exploration ability of algorithms, but it also weakens their local exploitation ability, and vice versa. Therefore, based on the characteristics of the problem and the different stages of algorithm evolution, exploring an efficient control strategy for the number and distribution of subswarms to ensure appropriate diversity in the overall population is worth further research. In addition, the advantages and disadvantages of these three learning strategies and their management modes for different fitness landscapes are also a promising research direction, which helps to design algorithms with better reliability and robustness for different problems.