Learning Competitive Swarm Optimization

Abstract

Particle swarm optimization (PSO) is a popular method widely used in solving different optimization problems. Unfortunately, in the case of complex multidimensional problems, PSO encounters some troubles associated with the excessive loss of population diversity and exploration ability. This leads to a deterioration in the effectiveness of the method and premature convergence. In order to prevent these inconveniences, in this paper, a learning competitive swarm optimization algorithm (LCSO) based on the particle swarm optimization method and the competition mechanism is proposed. In the first phase of LCSO, the swarm is divided into sub-swarms, each of which can work in parallel. In each sub-swarm, particles participate in the tournament. The participants of the tournament update their knowledge by learning from their competitors. In the second phase, information is exchanged between sub-swarms. The new algorithm was examined on a set of test functions. To evaluate the effectiveness of the proposed LCSO, the test results were compared with those achieved through the competitive swarm optimizer (CSO), comprehensive particle swarm optimizer (CLPSO), PSO, fully informed particle swarm (FIPS), covariance matrix adaptation evolution strategy (CMA-ES) and heterogeneous comprehensive learning particle swarm optimization (HCLPSO). The experimental results indicate that the proposed approach enhances the entropy of the particle swarm and improves the search process. Moreover, the LCSO algorithm is statistically and significantly more efficient than the other tested methods.

1. Introduction

Swarm intelligence (SI) is a branch of artificial intelligence (AI) based on the social behavior of simple organisms occurring in natural environments [1]. The source of its inspiration was observations of the collective behavior of animals such as birds, fishes, bees, bacteria, ants, squirrels and others [2,3,4,5,6,7,8]. There is a number of methods based on swarm intelligence. One of them is particle swarm optimization (PSO).

PSO was developed by Kennedy and Eberhart [9] as a stochastic method of optimization. The standard PSO is based on a swarm of particles, each of which wanders through the search space to find better solutions. The key to the success of the method is the ability to share information found by population individuals. Due to many advantages (simplicity, easy implementation, lack of coding and special operators) [4], the PSO method has been widely applied in solving various optimization problems, including control systems [10], prediction problems [11], image classification [12], energy management [13], bilevel programming problems [14,15], antenna design [16], scheduling problems [17,18], electromagnetism [19,20] and many others.

Unfortunately, in cases of complex, high-dimensional optimization problems with many local optima, PSO can encounter some difficulties associated with preserving a sufficient diversity of particles and maintaining a balance between exploration and exploitation. This leads to premature convergence and renders the results unsatisfactory. In order to avoid these inconveniences, various improved versions of the PSO have been developed. The improvements rely on [21]:

• Adjustment of parameters. Shi and Eberhart [22] indicated that the performance of the PSO method depends primarily on the inertia weight. In their opinion, the best results give inertial weight, which decreases linearly from 0.9 to 0.4. Zhang et al. [23] and Niu et al. [24] proposed to use a random inertia weight. In contrast to Zhang et al. [23] and Niu et. al. [24], Clerc [25] suggested that all coefficients should rather be constant and proved that inertia weight w = 0.729 and factors c1 = c2 = 1.494 can increase the rate of convergence. Venter and Sobieszczański [26] found that the PSO method is more effective when the acceleration coefficients are constant but different. According to them, the social coefficient should be greater than the cognitive one, and they proved that c2 = 2.5 and c1 = 1.5 produce superior performance. Nonlinear, dynamically changing coefficients were proposed by Borowska [27]. In this approach, the values of coefficients were affected by the performance of PSO and the number of iterations. In order to more efficiently control the searching process, Ratnaweera et al. [28] recommended time-varying coefficients (TVACs). An approach based on fuzzy systems was proposed by Chen [29].
• Topology. Topological structure has a great influence on the performance of the PSO algorithm. According to Kennedy and Mendes [30], a proper topology significantly improves the exploration ability of PSO. Lin et al. [31] and Borowska [32] indicated that the ring topology can help maintain swarm diversity and improve the algorithm’s adaptability. An approach based on multi-swarm structure was proposed by Chen et al. [33] and Niu [24]. In turn, a two-layer cascading structure was recommended by Gong et al. [34]. To alleviate premature convergence, Mendes et al. [35] introduced a fully informed swarm in which particles are updated based on the best locations of their neighbors. A PSO variant with adaptive time-varying topology connectivity (PSO-ATVTC) was developed by Lim et al. [36]. Carvalho et al. [37] proposed a particle topology based on the clan structure. The dynamic Clan PSO topology was described by Bastos-Filho et al. [38]. Shen et al. [39] proposed a multi-stage search strategy supplemented by mutual repulsion and attraction among particles. The proposed algorithm increases the entropy of the particle population and leads to a more balanced search process.
• Combining PSO with other methods. In order to obtain higher-quality solutions and enhance the performance of PSO, in many papers, researchers merge two or more different methods or their advantageous elements. Ali et al. [40] and Sharma et al. [41] combined PSO with genetic operators. A modified particle swarm optimization algorithm with simulated annealing strategy (SA) was proposed by Shieh et al. [42]. A PSO method with ant colony optimization (ACO) was developed by Holden et al. [43]. Cooperation of many swarms and four other methods for improving the efficiency of PSO was applied by Liu and Zhou [44]. Not only did the authors combine multi-population-based particle swarm optimization with the simulated annealing method but also with co-evolution theory, quantum behavior theory and mutation strategy. A different approach was presented by Cheng et al. [45]. To improve the exploration ability of PSO, they used a multi-swarm framework combining the feedback mechanism with the convergence strategy and the mutation strategy. The proposed approach helps reach a balance between exploration and exploitation and reduces the risk of premature convergence.
• Adaptation of learning strategy. This approach allows particles to acquire knowledge from high-quality exemplars. In order to increase the adaptability of PSO, Ye et al. [46] developed dynamic learning strategy. A comprehensive learning strategy based on historical knowledge about particle position was recommended by Liang et al. [47] and was also developed by Lin et al. [48]. Instead of individual learning of particles based on their historical knowledge, Cheng and Jin [49] introduced a social learning mechanism using sorting of swarm and learning from demonstrators (any better particles) of the current swarm. The method turned out to be effective and computationally efficient. A learning strategy with operators of the genetic algorithm (GA) and a modified updating equation based on exemplars was proposed by Gong et al. [34]. To enhance diversity and improve the efficiency of PSO, Lin et al. [31] merged PSO with genetic operators and also connected them with global learning strategy and ring topology. Learning strategy with genetic operators and interlaced ring topology was also proposed by Borowska [32]. To improve the searching process, Niu et al. [50] recommended applying learning multi-swarm PSO based on a symbiosis.

In order to improve the performance of the particle swarm optimization method, Cheng et al. [21] introduced a competitive swarm optimizer (CSO) based on PSO. In CSO, neither the personal best position of particles nor the global best position is required. Instead of them, a simple pairwise competition mechanism within one single swarm was introduced. Particles do not need knowledge about their historical positions as they learn only from the winner.

Unfortunately, although the CSO method has better search capability than traditional PSO, it does not always perform well and obtain expected results for complex optimization problems. Difficulties are associated with the loss of population diversity too quickly and maintaining a balance between exploration and exploitation. This leads to a deterioration in the effectiveness of the method and premature convergence.

To reduce these inconveniences, ensure diversity of particles and limit the risk of getting stuck in the local optimum, in this paper, a new learning competitive swarm optimization called LCSO is presented. The proposed approach is based on the particle swarm optimization method (PSO) and a competition concept. In LCSO, particles do not use information about their personal best position and global best particle in the swarm; instead of that, the competition mechanism was applied but in a different way than in CSO. In LCSO, the swarm is divided into sub-swarms, each of which can work independently. In each sub-swarm, three particles participate in the tournament. The participants who lost the tournament learn from their competitors. The winners take part in the tournament between sub-swarms. The new algorithm was examined on a set of test functions. To evaluate the effectiveness of the proposed LCSO, the test results were compared with those achieved through the competitive swarm optimizer (CSO) [21], comprehensive particle swarm optimizer (CLPSO) [47], PSO [51], fully informed particle swarm (FIPS) [35], the covariance matrix adaptation evolution strategy (CMA-ES) [52] and heterogeneous comprehensive learning particle swarm optimization (HCLPSO) [53].

2. The PSO Method

As mentioned in the introduction, particle swarm optimization is a method based on swarm intelligence and the collective behavior of animal societies. It was first proposed by Kennedy and Eberhart [9] as a new simple optimization tool. Because of its effectiveness, it has been regarded as a powerful method of optimization and has become a competitive approach to the genetic algorithm and other artificial intelligence tools [1].

In PSO, the optimization process is performed by the population of individuals called a swarm of particles. The swarm consists of N particles that move in the D-dimensional search space. The individual particle within the swarm can be considered as a point of the search space determined by two vectors: X_i = (x_i1, x_i2, …, x_iD) named a position vector and V_i = (v_i1, v_i2, …, v_iD) named a velocity vector. Both vectors are randomly generated at the first step of the PSO algorithm. Each particle roams through the search space according to its velocity vector V_i and remembers its personal best position pbest_i = (pbest_i1, pbest_i2, …, pbest_iD) that it found during its way and the best position gbest = (gbest₁, gbest₂, …, gbest_D) found in an entire swarm. The particles are also evaluated on their quality, which is measured based on the objective function of the optimized task. The particle wandering in the search space is performed according to the velocity equation determined as follows:

$$V_i\left(t+1\right)=w\cdot V_i\left(t\right)+c_1\cdot r_1\left(pbes{t}_i-X_i\left(t\right)\right)+c_2\cdot r_2\left(gbest-X_i\left(t\right)\right)$$

(1)

The change of the particle position is realized by adding the velocity vector to its position vector according to the given equation:

$$X_i\left(t+1\right)=X_i\left(t\right)+V_i\left(t+1\right)$$

(2)

where t is the iteration number, w means the inertia weight (that determines the impact of the previous velocity of the particle on its current velocity), factors c₁ and c₂ are acceleration coefficients, and r₁ and r₂ represent two random numbers uniformly distributed between 0 and 1. The pseudo code of the standard PSO method is presented in Algorithm 1.

Algorithm 1 Pseudo code of the PSO algorithm.

Determine the size of the swarm

for j = 1 to size of the swarm do

Generate an initial position and velocity of the particle,

Evaluate the particle;

Determine the pbest (personal best position) of the particle

end

Select the gbest (the best position) found in the swarm

while termination criterion is not met do

for j = 1 to size of the swarm do

Update the velocity and position of each particle according to Equations (1) and (2)

Evaluate new position of the particle;

Update the pbest (personal best position) of each particle

end

Update the gbest (the best position) found in the swarm

end

3. The Proposed LCSO Algorithm

The proposed LCSO (learning competitive swarm optimization) is a two-stage-learning-based method, which combines particle swarm optimization with a competition concept. In the first stage, particles take part in the tournaments within sub-swarms. In the second stage, tournaments are held between sub-swarms. Although LCSO is based on PSO, the knowledge acquisition mechanism is different. In LCSO, particles do not store any historical information about both their personal best position and global best particle in the swarm. Instead, particles derive knowledge from their better competitors.

In the first stage of LCSO, the swarm of N particles is divided into p sub-swarms. In every iteration, each particle within a sub-swarm participates in a tournament. In the sub-swarms, the tournaments are organized independently. In one tournament, participate 3 randomly selected particles. In one iteration, a particle takes part in a competition only once. The best particle of the tournament (winner) goes to the next iteration without updating. The particle that has finished the tournament as a second one (runner-up) learns from the winner according to the equations:

$$V_s\left(t+1\right) =r_1\cdot V_s\left(t\right) + r_2\cdot \left(X_w\left(t\right)-X_s\left(t\right)\right)$$

(3)

$$X_s\left(t+1\right) =X_s\left(t\right) + V_s\left(t+1\right)$$

(4)

The particle that took the last place (loser) learns from its competitors according to the following formulas:

$$V_l\left(t+1\right) =r_1\cdot V_l\left(t\right) + r_2\left(X_w\left(t\right)-X_l\left(t\right)\right) + r_3\left(X_s\left(t\right)-X_l\left(t\right)\right)$$

(5)

$$X_l\left(t+1\right) =X_l\left(t\right) + V_l\left(t+1\right)$$

(6)

where X_w is the position of the best particle (winner), V_s and X_s are velocity and position of the particle that got second place (runner-up), V_l and X_l are velocity and position of the particle that got last place (loser), t is the iteration number, and r₁, r₂ and r₃ are randomly generated numbers in the range [0, 1].

This means that for the sub-swarm of M particles, only two out of three particles are updated, whereas one in three particles of each sub-swarm go to the next stage without updating. Then the tournaments are organized between sub-swarms. For each sub-swarm, one particle from the set of the winners is selected. Subsequently, the (each three) particles participate in the tournament. The particle that finished the tournament as a second one (runner-up) learns from the winner according to Equations (3) and (4). The particle that got last place (loser) learns from its competitors according to (5) and (6). Next, the tournaments are organized again in sub-swarms. Figure 1 illustrates the concept of learning LCSO.

This means that among particles that participate in competition between sub-swarms only two out of three particles are updated and one-third of them go to the next iteration without updating.

The proposed sub-swarms topology limits the excessive loss of swarm diversity and helps maintain a balance between exploration and exploitation. The competition strategy (in the sub-swarm) promotes the interaction among particles of the sub-swarm. The introduced learning strategy in the sub-swarm ensures the exchange of information between them. The tournament strategy among the winners ensures sharing of information between sub-swarms. In this way, the sub-swarm that falls into the local optimum has a chance to jump out of it by learning from the winners of other sub-swarms.

The pseudo code of the LCSO method is summarized in Algorithm 2.

Algorithm 2 Pseudo code of the LCSO algorithm. FEs is the number of fitness evaluations. The termination condition is the maximum number of fitness evaluations (maxFEs).

Determine the size of the swarm;

Determine the number of sub-swarms;

Determine number of particles in the sub-swarms;

Randomly initialize position and velocity of the particles in sub-swarms;

t = 0;

while termination criterion (FEs ≤ maxFEs) is not met do

Evaluate the particles fitness;

Parallel in each sub-swarm organize tournament independently:

while sub-swarm ≠ Ø do

Randomly select three particles, compare their fitness and determine winner Xw, runner-up Xr and loser Xl

Update runner-up’s position according to (3) and (4)

Update loser’s position according to (5) and (6)

end

Organize tournament between sub-swarms:

Randomly select one particle from the winners of each sub-swarm

while set of selected winners ≠ Ø do

Randomly select three particles

compare fitness of the selected particles to determine winner Xw, runner-up Xr and loser Xl

Update runner-up’s position according to (3) and (4)

Update loser’s position according to (5) and (6)

end

t = t + 1;

end

output the best solution

4. Results

The tests of the proposed algorithm were performed on a set of benchmark functions, sixteen of which are presented in this article and described in

Table 1. Test functions.

Function	Formula	Fmin	Range
Sphere	$f_1={\displaystyle {\displaystyle \sum }_{i=1}^n}{x}_i{}^2$	0	[−100, 100]n
Schwefel 1.2	$f_2={\displaystyle {\displaystyle \sum }_{i=1}^n}({\displaystyle {\displaystyle \sum }_{j=1}^i}{x}_j{)}^2$	0	[−100, 100]n
Quartic	$f_3={\displaystyle {\displaystyle \sum }_{i=1}^n}i{x}_i^4+random\left[0,1\right)$	0	[−100, 100]n
Schwefel 2.22	$f_4={\displaystyle {\displaystyle \sum }_{i=1}^n}|x_i|+{\displaystyle {\displaystyle \prod }_{i=1}^n}\left|x_i\right|$	0	[−10; 10]n
Rosenbrock	$f_5={\displaystyle {\displaystyle \sum }_{i=1}^{n-1}}[100{\left(x_{i+1}-x_i^2\right)}^2+{\left(x_i-1\right)}^2$	0	[−30, 30]n
Griewank	$f_6=\frac{1}{4000}{\displaystyle {\displaystyle \sum }_{i=1}^n}{x}_i^2-{\displaystyle {\displaystyle \prod }_{i=1}^n}\cos \left(\frac{x_i}{\sqrt{i}}\right)+1$	0	[−600, 600]n
Rastrigin	$f_7={\displaystyle {\displaystyle \sum }_{i=1}^n}\left(x_i^2-10\cos \left(2\pi x_i\right)+10\right)$	0	[−5.12, 5.12]n
Ackley	$f_8=-20 \exp \left(-0.2\sqrt{\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}{x}_i^2}\right)$ $-\exp \left(\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}\cos (2\pi \ast x_i)\right)+20+e$	0	[−32, 32]n
Zakharov	$f_9=-{\displaystyle {\displaystyle \sum }_{i=1}^n}{x}_i+({\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{i}{2}{x}_i{)}^2+({\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{i}{2}{x}_i{)}^4$	0	[−10, 10]n
Schwefel	$f_{10}=418.9829 · n-{\displaystyle {\displaystyle \sum }_{i=1}^n}(x_i·\sin (|x_i{|}^{0.5})$	0	[−400, 400]n
Weierstrass	$f_{11}={\displaystyle {\displaystyle \sum }_{i=1}^n}\left({\displaystyle {\displaystyle \sum }_{k=0}^{kmax}}[a^k\cos \left(2\pi b^k\left(x_i+0.5\right)\right)]\right)$ $\hspace{1em}\hspace{1em}\hspace{1em}-n{\displaystyle {\displaystyle \sum }_{k=0}^{kmax}}[a^k\cos \left(2\pi b^k·0.5\right)]$     a = 0.5, b = 3, kmax = 20	0	[−0.5, 0.5]n
Rotated Rastrigin	$f_{12}={\displaystyle {\displaystyle \sum }_{i=1}^n}\left(z_i^2-10\cos \left(2\pi z_i\right)+10\right),$ $z=x · M$	0	[−5.12, 5.12]n
Rotated Griewank	$f_{13}=\frac{1}{4000}{\displaystyle {\displaystyle \sum }_{i=1}^n}{z}_i^2-{\displaystyle {\displaystyle \prod }_{i=1}^n}\cos \left(\frac{z_i}{\sqrt{i}}\right)+1,$ $z=x · M$	0	[−600, 600]n
Rotated Schwefel	$f_{14}=418.9829 · n-{\displaystyle {\displaystyle \sum }_{i=1}^n}{y}_i$ $y_i=\left\{\begin{array}{c}{x}_i\sin ({\left|x_i\right|}^{0.5} if\left|x_i\right|\le 500\\ 0.001{\left(\left|x_i\right|-500\right)}^2 if\left|x_i\right|>500\end{array}\right.$ $for i=1,2\dots .n, x=x^{\prime }+420.96$ $\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{x}^{\prime }=M\left(z-420.96\right)$	0	[−400, 400]n
Rotated Weierstrass	$f_{15}={\displaystyle {\displaystyle \sum }_{i=1}^n}\left({\displaystyle {\displaystyle \sum }_{k=0}^{kmax}}[a^k\cos \left(2\pi b^k\left(x_i+0.5\right)\right)]\right)$ $\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}-n{\displaystyle {\displaystyle \sum }_{k=0}^{kmax}}[a^k\cos \left(2\pi b^k·0.5\right)]$    a = 0.5, b = 3, kmax = 20, $y=M · x$	0	[−0.5, 0.5]n
Rotated Ackley	$f_{16}=-20 \exp \left(-0.2\sqrt{\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}{x}_i^2}\right)$ $-\exp \left(\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}\cos (2\pi \ast x_i)\right)+20+e$ $z=x · M$	0	[−32, 32]n

.

The first five test functions (f₁–f₅) are unimodal, the next six (f₆–f₁₁) are multimodal, whereas the remaining ones (f₁₂–f₁₆) are rotated multimodal functions. The performance of the presented LCSO algorithm for all functions was compared with the results using the competitive swarm optimizer (CSO) [21], comprehensive particle swarm optimizer (CLPSO) [47], particle swarm optimization (PSO) [51], fully informed particle swarm (FIPS) [35], covariance matrix adaptation evolution strategy (CMA-ES) [52] and heterogeneous comprehensive learning particle swarm optimization (HCLPSO) [53].

The experiments were conducted on a PC with an Intel Core i7-3632QM 2.2 GHz CPU and Microsoft Windows 10 64-bit operating system. The LCSO was implemented in C with Microsoft Visual Studio 2010 Enterprise.

The parameter settings of the comparison algorithms are listed in

Table 2. Parameter settings of algorithms.

Algorithm	Inertia Weight w	Acceleration Coefficients c1, c2, c	Other Parameters
CLPSO	w = 0.9–0.4	c = 1.496	-
FIPS		c = 2.05	χ = 0.729
PSO	w = 0.9–0.4	c1 = 2.0, c2 = 2.0	-
HCLPSO	w = 0.99–0.2	c1 = 2.5–0.5, c2 = 0.5–2.5, c = 3–1.5	-
CSO	w = 0.7298	c = 1.49618	pm = 0.01, sg = 7

. The details of algorithms used for comparison can be found in [21,35,47,51,53].

For all tested functions, the experiments with dimension size D = 30 were conducted. The population size of all algorithms is N = 72, whereas the maximum number of evaluations is 10,000. The number of sub-swarms p = 3. The exemplary results of the tests are summarized in

Table 3. Comparison results.

Function		CLPSO	PSO	HCLPSO	CSO	CMA-ES	FIPS	LCSO
f1	Mean	4.15E-06	2.43E-17	8.93E-10	3.22E-15	4.16E-19	1.03E-06	5.24E-21
	Std	3.81E-06	3.15E-17	7.11E-10	3.75E-15	4.07E-19	1.59E-06	2.19E-21
f2	Mean	2.43E-01	6.54E-03	8.93E-02	3.22E-02	1.78E-03	4.88E-00	5.63E-05
	Std	3.15E-01	4.81E-03	7.11E-02	3.75E-02	4.02E-04	5.37E-00	4.39E-05
f3	Mean	3.42E-03	1.34E-02	2.33E-03	5.22E-04	5.21E-01	2.41E+03	3.61E-06
	Std	5.17E-03	9.06E-03	7.94E-04	5.08E-04	2.79E-03	1.47E+03	6.42E-06
f4	Mean	5.16E-08	3.44E-14	4.33E-06	4.88E-17	3.25E-07	3.17E-07	4.86E-17
	Std	1.04E-07	2.80E-14	1.13E-05	4.63E-16	2.81E-07	4.54E-07	3.40E-16
f5	Mean	3.78E+01	6.14E+01	3.53E+01	4.20E+01	1.02E+01	2.82E+01	2.54E+00
	Std	1.86E+01	5.80E+01	5.14E+00	3.15E+01	1.61E+00	2.14E+01	5.72E+00
f6	Mean	4.43E-06	1.45E-02	4.61E-08	7.21E-10	3.34E-09	2.93E-03	3.98E-12
	Std	1.01E-05	1.29E-02	5.42E-08	5.36E-10	1.02E-10	4.18E-03	3.41E-11
f7	Mean	1.34E+00	4.38E+01	4.78E-04	3.80E-04	1.8E+01	7.32E+01	6.54E-06
	Std	6.16E-01	8.75E+00	7.05E-05	2.52E-04	7.35E-01	2.25E+01	3.77E-05
f8	Mean	5.12E-01	2.24E+00	1.15E-09	4.60E-10	7.35E-12	5.74E+00	8.12E-08
	Std	8.39E-01	1.46E+00	3.67E-10	1.88E-09	1.14E-12	6.18E-01	7.93E-09
f9	Mean	3.97E-01	1.56E+00	6.30E-02	5.61E-02	1.39E-01	2.25E-01	3.08E-03
	Std	3.12E-01	1.25E+00	5.47E-02	3.95E-02	2.61E-02	2.43E-01	2.95E-03
f10	Mean	1.71E-03	1.86E+03	2.08E-03	6.53E+03	1.25E-03	6.62E+02	3.15E-05
	Std	1.59E-03	2.07E+02	1.37E-03	3.87E+03	1.94E-03	4.94E+02	4.39E-05
f11	Mean	2.39E-05	1.89E-03	4.38E-04	3.15E-03	2.32E-03	5.29E-04	5.37E-07
	Std	2.12E-05	1.67E-03	3.65E-04	1.97E-03	1.85E-03	1.92E-04	3.71E-07
f12	Mean	1.89E+02	5.94E+01	5.98E+01	1.92E+01	2.74E+01	2.21E+02	7.67E+00
	Std	5.14E+01	6.73E+00	4.26E+01	5.07E+00	1.30E-01	2.80E+01	4.02E+00
f13	Mean	3.44E-03	7.68E-02	7.53E-03	5.41E-09	2.28E-08	6.67E-04	9.16E-12
	Std	4.96E-03	8.39E-02	8.66E-03	7.92E-09	3.46E-09	1.02E-03	7.85E-11
f14	Mean	6.23E+03	4.66E+03	6.83E+03	7.78E+03	1.15E+03	7.65E+03	4.59E+02
	Std	4.18E+03	3.19E+03	3.99E+03	3.54E+03	3.29E+03	3.61E+03	6.34E+02
f15	Mean	3.27E+01	2.55E+01	6.45E+01	1.83E-01	2.12E+01	4.07E+01	3.47E-03
	Std	3.39E+01	3.12E+01	4.73E+01	2.78E-01	1.96E+00	8.65E+00	2.51E-03
f16	Mean	8.27E-01	2.25E+00	2.31E+00	5.11E-10	1.76E+00	2.71E-03	5.09E-14
	Std	6.54E-02	6.58E-01	5.15E-02	5.16E-10	5.04E-01	2.03E-03	6.71E-14

, and the presented values were averaged over 32 runs. The best results are shown in bold.

To facilitate understanding of the results, the comparison of the algorithm’s effectiveness on a logarithmic scale is shown in Figure 2 and Figure 3. In order to compare the convergence rate of the tested algorithms, the convergence curves on six representative functions were plotted and presented in Figure 4 and Figure 5. To evaluate the effectiveness of the proposed method, a statistical t-test was conducted. For all comparisons, a confidence level of 0.05 was used. The t-values between LCSO and the other considered algorithms for 30 dimensions are presented in

Table 4. The t-test results.

Function	t-Value between CLPSO and LCSO	t-Value between PSO and LCSO	t-Value between HCLPSO and LCSO	t-Value between CSO and LCSO	t-Value between FIPS and LCSO	Two-Tailed p between CLPSO and LCSO	Two-Tailed p between PSO and LCSO	Two-Tailed p between HCLPSO and LCSO	Two-Tailed p between CSO and LCSO	Two-Tailed p between FIPS and LCSO
f1	6.16E+00	4.36E+00	7.10E+00	4.86E+00	3.66E+00	1.60E-08	3.18E-05	1.95E-10	4.51E-06	4.02E-04
f2	4.36E+00	7.62E+00	7.10E+00	4.85E+00	5.14E+00	3.18E-05	1.59E-11	1.99E-10	4.67E-06	1.40E-06
f3	3.74E+00	8.36E+00	1.66E+01	5.77E+00	9.27E+00	3.12E-04	4.21E-13	3.50E-30	9.19E-08	4.55E-15
f4	2.82E+00	6.94E+00	2.16E+00	N/A	3.95E+00	5.89E-03	4.28E-10	3.30E-02	N/A	1.47E-04
f5	1.02E+01	5.71E+00	2.41E+01	6.97E+00	6.55E+00	3.47E-17	1.19E-07	6.31E-43	3.67E-10	2.64E-09
f6	2.49E+00	6.36E+00	4.81E+00	7.55E+00	3.97E+00	1.45E-02	6.49E-09	5.44E-06	2.26E-11	1.40E-04
f7	1.23E+01	2.83E+01	6.95E+00	4.66E+00	1.84E+01	1.35E-21	5.57E-49	4.01E-10	1.00E-05	1.45E-33
f8	3.45E+00	8.68E+00	N/A	N/A	5.25E+01	8.23E-04	8.84E-14	N/A	N/A	1.35E-73
f9	7.14E+00	7.04E+00	6.19E+00	7.57E+00	5.17E+00	1.63E-10	2.60E-10	1.43E-08	2.05E-11	1.26E-06
f10	5.97E+00	5.08E+01	8.45E+00	9.55E+00	7.58E+00	3.81E-08	3.10E-72	2.70E-13	1.18E-15	1.97E-11
f11	6.23E+00	6.40E+00	6.78E+00	9.04E+00	1.56E+01	1.15E-08	5.35E-09	9.11E-10	1.44E-14	3.01E-28
f12	1.93E+01	3.61E+01	6.67E+00	9.76E+00	4.13E+01	4.12E-35	1.84E-58	1.51E-09	4.01E-16	8.48E-64
f13	3.80E+00	5.01E+00	4.76E+00	3.75E+00	3.58E+00	2.52E-04	2.38E-06	6.62E-06	3.00E-04	5.39E-04
f14	7.48E+00	7.07E+00	8.64E+00	1.11E+01	1.07E+01	3.26E-11	2.25E-10	1.09E-13	3.95E-19	2.93E-18
f15	5.28E+00	4.48E+00	7.47E+00	3.54E+00	2.58E+01	7.69E-07	2.06E-05	3.39E-11	6.20E-04	1.94E-45
f16	6.93E+01	1.87E+01	2.46E+02	5.42E+00	7.31E+00	4.80E-85	3.78E-34	1.53E-138	4.21E-07	7.22E-11

.

The results of the tests indicate that the LCSO method with the proposed approach is effective as it achieved superior performance over the other tested algorithms.

For the unimodal functions, LCSO obtained the best results (although the standard deviation values were not always the lowest) among all the algorithms, and only for f₄, the results of LCSO and CSO were comparable, but LCSO turned out to be more stable. The weakest results for the f₁ function were obtained by CLPSO, for f₂ and f₃ by FIPS and for f₅ by PSO. For multimodal functions, in the case of f₆, f₉, f₁₀, f₁₁, f₁₂, f₁₄, f₁₅ and f₁₆, the LCSO method also achieved the best outcomes and was more stable than the other tested algorithms. In the case of f₇ and f₁₃ functions, LCSO achieved higher performance compared to the other methods, but it was not as stable as them. In the case of f₇, the HCLPSO method turned out to be more stable than the others. For f₁₃ function, the CSO method was more stable than LCSO. In the case of f₈ function, LCSO performed worse than CSO and HCLPSO but much better than CLPSO, PSO and FIPS. In the case of f₁₆ function, the results gained by CSO are also much better than those achieved by CLPSO, PSO, FIPS and even HCLPSO but not as good as LCSO. In all the tests, the least effective was PSO as it obtained the poorest results. Particles of PSO moved irregularly and had a tendency to fall and stop in the local minima. The convergence curves presented in Figure 4 and Figure 5 indicate that, for unimodal functions, LCSO converged slower in the early stage than most of the compared methods. At this stage, CSO was found to perform with the highest rate. However, after about 3000 iterations, LCSO surpasses CSO and other tested methods. Regarding multimodal functions, at an early stage, LCSO also converges slower than CSO but faster than CLPSO, PSO, FIPS and HCLPSO. In the middle stage, LCSO surpasses CSO and converges faster than the other algorithms (except f₈), but it needs as many as 6000 iterations to achieve this. However, for f₈ function, LCSO slows down in the middle stage and ultimately converges slower than CSO and HCLPSO.

The obtained results confirm that the proposed method is effective and efficient. The advantages of the proposed LCSO method include:

• The use of sub-swarms helps maintain the diversity of the population and keep the balance between the global exploration and local exploitation;
• Particles learning from the winners can effectively search space for a better position;
• Good information found by sub-swarm is not lost;
• Particles can learn from the useful information found by other sub-swarms;
• In each iteration, the position and velocity of only two out of three particles is updated which significantly reduces the cost of computations;
• Particles do not need to remember their personal best position; instead, the competition mechanism is applied;
• LCSO can obtain better results and convergence than the other algorithms.

Based on the t-test results summarized in Table 4, it can be concluded that the performance of the proposed learning competitive swarm optimization (LCSO) algorithm is significantly better than the other methods with a 95% confidence level in a statistically meaningful way.

In order to assess the effectiveness of the LCSO method for a larger dimension of the search space, functions with dimensions of 100, 500 and 1000 were investigated. The tests were performed with a population of N = 99 particles. The maximum number of evaluations was 80,000. The results of the tests were averaged over 10 runs and are presented in

Table 5. Results on 100, 500 and 1000 dimensional functions.

Function	D = 100	D = 500	D = 1000
f1	1.97E-23	2.03E-11	2.14E-09
f2	4.62E-01	7.39E+01	2.17E+02
f3	1.95E-01	1.38E+04	2.01E+05
f4	5.16E-02	8.41E+00	3.56E+02
f5	9.38E+01	5.22E+02	9.72E+02
f6	1.02E-18	5.46E-12	3.65E-10
f7	1.06E-13	3.71E-08	5.29E-06
f8	5.12E-14	2.24E-05	4.15E-03
f9	7.06E-07	4.67E-04	5.07E-02
f10	3.18E+04	1.82E+05	3.94E+05
f11	1.37E-02	2.56E+01	4.61E+03
f12	1.25E-12	3.77E-09	6.29E-08
f13	1.03E-14	6.32E-09	5.07E-08
f14	2.56E+04	1.69E+05	3.83E+05
f15	1.42E+01	3.70E+02	7.26E+02
f16	2.07E-13	3.24E-07	4.19E-06

.

It should be noted that, despite the increase of the search space dimensions, the proposed LCSO method is still effective for functions f₁, f₆, f₇, f₈, f₉, f₁₂, f₁₃ and f₁₆. An increase in the number of dimensions requires an increase in the size of the population. It also entails an increase in computation costs.

5. Conclusions

In this paper, a new learning competitive swarm optimization algorithm (LCSO) based on the particle swarm optimization method and competition mechanism was proposed. In LCSO, particles do not have to remember their personal best positions; instead of that, they participate in the tournament. The tournaments take place in sub-swarms as well as between them. The participants of the tournament update their positions by learning from their competitors.

The efficiency of the LCSO method was tested on a set of benchmark functions. Then the test results were compared with four different variants of PSO, including CSO, CLPSO, PSO, CMA-ES, FIPS and HCLPSO. The obtained results indicate that the proposed LCSO is faster and more effective than the other examined algorithms. The competition mechanism used in LCSO helps maintain particle diversity and improve its exploration ability.

Future work will focus on the implementation of the LCSO algorithm in real-world optimization problems. The application of the LCSO method in machine learning and image compression is also a promising area of research.