An Overview of Variants and Advancements of PSO Algorithm

Abstract

Particle swarm optimization (PSO) is one of the most famous swarm-based optimization techniques inspired by nature. Due to its properties of flexibility and easy implementation, there is an enormous increase in the popularity of this nature-inspired technique. Particle swarm optimization (PSO) has gained prompt attention from every field of researchers. Since its origin in 1995 till now, researchers have improved the original Particle swarm optimization (PSO) in varying ways. They have derived new versions of it, such as the published theoretical studies on various parameters of PSO, proposed many variants of the algorithm and numerous other advances. In the present paper, an overview of the PSO algorithm is presented. On the one hand, the basic concepts and parameters of PSO are explained, on the other hand, various advances in relation to PSO, including its modifications, extensions, hybridization, theoretical analysis, are included.

1. Introduction

In today’s complex life, everyone knowingly and unknowingly takes decisions that may be related to single or multiple objectives. The eventual goal of these decisions is to maximize the benefit or to minimize the loss. From all of the available options, we prefer to choose the best option. So, optimization is a process of finding as effective as possible an outcome, or it is an act of choosing the best option from among the available options with the objective of maximization or minimization and with or without any given set of constraints. It has several components, such as decision variables, constraints and objectives. It is a well-known topic among numerous scholars and research groups engaged in the development and advancement of any area of science, engineering, technology, mathematics, philosophy and many more. Attempts are always made to continuously improve the existing work. There is a long list of algorithms available in the literature. These can be classified as exact and approximate algorithms. Due to the limitations of the exact methods, research is more tilted towards the approximate methods. Approximate methods are mainly heuristic and metaheuristic. The metaheuristic algorithms are non-deterministic in nature.

Broadly, there are three main branches of metaheuristic algorithms: Evolutionary algorithms (EA); physics and chemistry-based algorithms; and swarm intelligence-based algorithms. Inspired by the principles of natural selection and natural genetics, the evolutionary algorithms (EA) are always famous with the research scientists. Some of the evolutionary algorithms (EA) are genetic algorithms, virulence optimization algorithms, differential search algorithms, biogeography-based algorithms, differential evolution algorithms, cultural algorithms, etc. The physics and chemistry-based algorithms are those algorithms that mimicked some physical and/or chemical laws i.e., their inspiration source is from physics and chemistry. Famous algorithms in the category of physics and chemistry algorithms are: big bang–big crunch, galaxy-based search algorithm, spiral optimization, electro-magnetism optimization, black hole, charged system search, etc. Swarm intelligence (SI) is a major branch of computational intelligence, based on studying the collective behavior of swarms in nature that interact with each other locally in the absence of any kind of supervision. Swarm intelligence is also termed as collective intelligence. Examples of SI include ant colonies, birds flocking, animal herding, bacterial growth, fish schooling, etc. Until now, a wide variety of research has been carried out on various swarm intelligence algorithms to solve different optimization problems (which has been elaborated in

Table 1. List of some of the swarm intelligence-based algorithms.

Name of Algorithm	Year	Description	Ref. No.
Artificial hummingbird algorithm	2022	Zhao et al. proposed an artificial hummingbird algorithm (AHA) to tackle optimization problems and proved its effectiveness over other metaheuristics with experimental results.	[1]
Chimp Optimization Algorithm (Khishe and Mosavi (2020a))	2022	Jia et al. presented an enhanced chimp optimization algorithm (EChOA) and analyzed its performance on 12 classical benchmark functions and 15 CEC2017 benchmark functions.	[2]
Rat swarm optimization (Dhiman et al. (2021))	2021	Dhiman et al. presents swarm-based rat swarm optimization and tested its performance on unimodal, multimodal and CEC-15 special session benchmark functions.	[3]
African Vulture’s Optimization Algorithm (Abdollahzadeh et al. (2021)	2021	A new metaheuristics, namely African Vulture’s Optimization Algorithm (AVOA) is proposed by Abdollahzadeh et al. They proved it as a best algorithm on 30 out of 36 benchmark functions.	[4]
Dragonfly optimization	2021	Bhardwaj and Kim proposed dragonfly node identification algorithm (DNIA) and evaluated its robustness and efficiency using statistical analysis, convergence rate analysis, Wilcoxon test, Friedman rank test, and analysis of variance on classical as well as modern IEEE CEC 2014 benchmark functions.	[5]
Horse herd optimization algorithm	2021	In order to solve high dimensional optimization techniques, MiarNaeimi et al. developed a new meta-heuristic algorithm called the Horse herd Optimization Algorithm (HOA).Through statistical results, they demonstrated the merits of their proposed algorithm.	[6]
Gaining-sharing knowledge-based algorithm	2020	Mohamed et al. proposed a gaining-sharing knowledge-based algorithm and proved it was better by completing experiments on various problems, along with CEC 2017 benchmark functions.	[7]
Coronavirus optimization algorithm	2020	Martinez-Alvarez et al. introduced a novel bio-inspired metaheuristic, based on the coronavirus behavior. They elaborated major advantages of coronavirus optimization algorithm compared to other similar strategies.	[8]
Harris Hawks Optimization	2019	Heidari et al. proposed a novel paradigm called Harris Hawks Optimizer (HHO), and tested it on 29 benchmark problems and several real-world engineering problems	[9]
African Buffalo Optimization	2015	Odili et al. developed a novel optimization technique, namely the African Buffalo Optimization (ABO) and checked its validation on a number of benchmark Traveling Salesman Problems. Authors recommended to use ABO to solve knapsack problems.	[10]

).

The list of swarm intelligence algorithms is too long to explain each and every algorithm, such as the particle swarm optimization (PSO), ant colony optimization (ACO), firefly algorithm (FA), honey bee algorithm (HBA), grey wolf optimization (GWO), bat algorithm (BA), krill herd algorithm (KHA), cuckoo search algorithm (CSA), flower pollination algorithm (FPA), lion optimization algorithm (LOA), salp swarm algorithm (SA), cat algorithm (CA) and many more to come.

The success of these algorithms in terms of the lesser number of iterations, less computational efforts, etc. is largely based on their parameter tuning and parametric control. These algorithms have a common objective of searching for a good quality solution using less computational effort. For this objective, according to the researchers, there should be a proper balance between the exploration and exploitation abilities of the algorithm. Exploration is the process of searching entirely new regions of a whole searching space, whereas exploitation is the process of searching only those regions of a search space those are near to the already visited and searched area. According to Eiben and Schippers [11], these concepts of exploration and exploitation abilities were not fully understandable because, on one hand, there was no accepted opinion among the researchers about exploration and exploitation, and on the other hand, an increase in one’s capability resulted in the weakness of the other one. Therefore, a proper balance is imperative between the exploration and exploitation capabilities for the success of any algorithm to solve the optimization problem. However, due to the stochastic nature of the optimization problems, it is very difficult to achieve a balance between these components.

Success in solving any kind of optimization problem depends upon the selection of an appropriate algorithm. This was also proved by Wolpert and Macready [12], in their theorem called as the “no free lunch” theorem, that no algorithm is perfect to solve every kind of optimization problem. So, the main concept is to choose wisely an appropriate optimization technique to solve a given hand -in optimization problems, with less computational effort and a higher rate of convergence performance. Kennedy and Eberhart [13] introduced an evolutionary algorithm particle swarm optimization (PSO) as an optimization technique, based on the concept of birds flocking, fish schooling and even human social behavior. The main idea and structure of the algorithm was inspired by evolutionary computation. Presently, it is considered to be one of the leading swarm intelligence algorithms that is widely used in hybrid techniques, due to its simplicity, capability of searching for the global optimum and higher rate of convergence.

2. Concept of Particle Swarm Optimization (PSO) Technique

The particle swarm optimization (PSO) technique is a well-known population-based metaheuristics technique to solve optimization problems. This algorithm simulates the social behavior of birds within the flock to attain the target of food. With the combination of both self and social experience, a swarm of birds approaches their target of food. They continuously update their position, according to their own best position and the best position of the entire swarm, and regroup themselves, resulting in an optimal formation. This social–psychological behavior of birds inspired Russell Elberhart (electrical engineer) and James Kennedy (social psychologist) to apply this principle of social interaction to problem solving. Kennedy and Eberhart [13] developed the particle swarm optimization (PSO) technique for the purpose of optimizing continuous non-linear functions. This nature-based swarm-intelligence algorithm works on iterations. It starts with a population (called a swarm) of candidates’ solutions. Here, each particle represents a potential solution to the given problem. In every iteration, the population is updated by updating the velocity and position of each individual. These updates are based on personal best value (pbest) and global best value (gbest). Eberhart and Kennedy called pbest and gbest the two basic values. In the pbest model, the particles are influenced by their own position, but in the gbest model, the position of the particles is influenced by the best position found by any member of the entire population. Then, accordingly, each particle will converge to this new position. In short, we can say pbest is the best position or location obtained so far by the individual itself. Gbest is the best position by any individual obtained so far in the entire population during the search process in the solution space.

2.1. Updating of Velocity and Position of Particle in the Swarm

2.1.1. Updating of Velocity

In this algorithm, the regulation of velocity is considered as a major feature, as it is the main mechanism used to move the position of a particle to search for an optimal solution in the search space. Eberhart used the maximum values of velocity, and discussed results for different values of velocity. The velocity of particle k in the swarm in the (i + 1)^th iteration, is updated, according to the following Equation (1):

$${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + \mathrm{1}) ={\mathrm{V}}_{\mathrm{k}}(\mathrm{i} ) + {\mathrm{c}}_{1 }{\mathrm{r}}_1\left({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{\mathrm{k}} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right) + {\mathrm{c}}_{2 }{\mathrm{r}}_2\left({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right)$$

(1)

where ${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1)$ is the velocity of particle k at the (i + 1)^th iteration; ${\mathrm{X}}_{\mathrm{k}}(\mathrm{i})$ is the position of particle k in the i^th iteration; ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{\mathrm{k}}$ is individual best position of the particle k in the i^th iteration; ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{}$ is the global best position of the any particle in the i^th iteration;${\mathrm{c}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{c}}_2$ are the real acceleration coefficients that control how much the global and individual best positions should influence the particle’s velocity; ${\mathrm{r}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{r}}_2$ are uniformly distributed random numbers in the range 0 and 1, used to maintain an adequate level of diversity in the population. There are three components of the velocity update Equation (1) of k^th particle in the (i + 1)^th iteration, namely:

• Momentum part;
• Cognitive part;
• Social part.

A brief description of the velocity equation has been illustrated in

Table 2. Details of the components of velocity update Equation (1).

Notation	Name of Component	Contribution of the Component in Updating the Velocity
${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} )$	Momentum part	It serves as a memory of the immediate past flight as it uses the previous velocity. It is also taken to be inertia component that makes a balance between the exploration and exploitation of each particle in search space.
${\mathrm{c}}_{1 }{\mathrm{r}}_1\left({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{\mathrm{k}} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right)$	Cognitive part	This cognitive part drives the particles to their own best position and is equivalent to the distance of the particle from its personal best position till now.
${\mathrm{c}}_{2 }{\mathrm{r}}_2\left({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right)$	Social Part	This is the social component of the velocity equation that drives the particle to the best position determined by the swarm.

.

2.1.2. Updating the Position of Particle

The position of each particle k, at every iteration (i + 1)^th, varies according to the following Equation (2):

$${\mathrm{X}}_{\mathrm{k}}(\mathrm{i} + 1) = {\mathrm{X}}_{\mathrm{k}}(\mathrm{i}) + {\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1)$$

(2)

where ${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1)$ is the velocity of particle k at the (i + 1)^th iteration; ${\mathrm{X}}_{\mathrm{k}}(\mathrm{i}+1)$ is the position of particle k in the (i + 1)^th iteration; ${\mathrm{X}}_{\mathrm{k}}(\mathrm{i})$ is the position of particle k in the i^th iteration.

Figure 1 illustrates the state of swarm at the iteration (i + 1)^th. It shows how the velocity of particle k is updated and that, in turn, updates the position of the k^th particle in the iteration (i + 1)^th.

2.2. Standard Particle Swarm Optimization (SPSO)

The authors Shi and Elberhart [14] introduced a new parameter $\omega$, called the inertia weight of a particle. With this new parameter, the velocity update Equation (1) changed to Equation (3):

$${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1) = \omega {\mathrm{V}}_{\mathrm{k}}(\mathrm{i} ) + {\mathrm{c}}_{1 }{\mathrm{r}}_1\left({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{\mathrm{k}} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right) + {\mathrm{c}}_{2 }{\mathrm{r}}_2({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{} - {\mathrm{X}}_{\mathrm{k}}\left(\mathrm{i}\right))$$

(3)

The particle swarm optimization (PSO) algorithm with this improvement is commonly referred to as the standard PSO.

2.3. Pseudocode of PSO Algorithm

The Pseudocde of the particle swarm optimization algorithm is illustrated in the Algorithm 1 as follows:

Algorithm 1: The Pseudocde of the algorithm is as follows

INPUT: Fitness function, lower bound, upper bound is the part of problem i.e., it will be given in the problem. Np (Population Size), I (No. of iteration), ω, ${\mathrm{c}}_1 , {\mathrm{c}}_2$ are to be chosen by the user. 1. Initialize a random population (P) and velocity (v) within the bounds; 2. Evaluate the objective function value (f) of P; 3. Assign ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{}$ as P and ${\mathrm{f}}_{\mathrm{p}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{}$ as f; 4. Identify the solution with best fitness and assign that solution as ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{}$ and fitness as ${\mathrm{f}}_{\mathrm{g}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} }$. For iteration:- for i = 1 to I          for k = 1 to Np Determine the velocity (${\mathrm{v}}_{\mathrm{k}}$) of kth particle; Determine the new position (${\mathrm{X}}_{\mathrm{k}}$) of kth particle; Bound ${\mathrm{X}}_{\mathrm{k}}$; Evaluate the objective function value ${\mathrm{f}}_{\mathrm{k}}$ of kth particle; Update the population by including ${\mathrm{X}}_{\mathrm{k}}$ and ${\mathrm{f}}_{\mathrm{k}}$; Update ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{k}} \mathrm{and} {\mathrm{f}}_{\mathrm{p}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} } \mathrm{if} {\mathrm{f}}_{\mathrm{k}} < {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{k}} } \mathrm{then} \{\begin{array}{c}{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{k}} = {\mathrm{X}}_{\mathrm{k}}\\ {\mathrm{f}}_{\mathrm{p}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} } = {\mathrm{f}}_{\mathrm{k}}\end{array}$; Update ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} \mathrm{and} {\mathrm{f}}_{\mathrm{g}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} } \mathrm{if} {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{k}} } < {\mathrm{f}}_{\mathrm{g}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} } \mathrm{then} \{\begin{array}{c}{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} = {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{k}} \\ {\mathrm{f}}_{\mathrm{g}\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} } = {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{k}} }{}_{}\end{array}$.            end end

2.4. Working Example of PSO

The PSO algorithm can be more understandable with a working example with some iterations. The example is illustrated as below:

Example: min $\mathrm{f}(\mathrm{x}) = {\mathrm{x}}_1^2 + {\mathrm{x}}_2^2 +{\mathrm{x}}_3^2 + {\mathrm{x}}_4^2 0\le {\mathrm{x}}_{\mathrm{i}} \le 10 \mathrm{i}=1,2,3,4$

Solution: Parameter setting of PSO algorithm

Population Size (${\mathrm{N}}_{\mathrm{p}}$) = 5, Inertia Weight (w) = 0.7, Acceleration Coefficient = ${\mathrm{c}}_1 = {\mathrm{c}}_2= 1.5$, Maximum iteration = 10

Iteration !: Generate random positions and velocities within domain $0\le {\mathrm{x}}_{\mathrm{i}} \le 10$ and evaluate fitness of positions i.e.,

$$\mathit{X}(\mathit{\text{position}})= \left[\begin{array}{cccc}4& 0& 0& 8\\ 3& 1& 9& 7\\ 0& 3& 1& 5\\ 2& 1& 4& 9\\ 6& 2& 8& 3\end{array}\right] \mathit{f}(\mathit{X})(\mathit{\text{fitness function}})= \left[\begin{array}{c}80\\ 140\\ 35\\ 102\\ 113\end{array}\right] \mathit{v}(\mathit{\text{velocity}})\left[\begin{array}{cccc}9& 6& 1& 8\\ 5& 1& 3& 0\\ 7& 4& 1& 4\\ 3& 0& 2& 1\\ 1& 6& 8& 7\end{array}\right]$$

Here fitness value of velocity is not required, fitness value of only positions is required

$${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{(1)} = \left[\begin{array}{cccc}4& 0& 0& 8\\ 3& 1& 9& 7\\ 0& 3& 1& 5\\ 2& 1& 4& 9\\ 6& 2& 8& 3\end{array}\right] {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = \left[\begin{array}{c}80\\ 140\\ 35\\ 102\\ 113\end{array}\right]$$

In the first iteration as we do not have any historical information, so current position is considered as best direction i.e., ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \mathrm{X}$.

Now, ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$

So, in iteration 1, we get:

$$\mathit{V}=\left[\begin{array}{cccc}9& 6& 1& 8\\ 5& 1& 3& 0\\ 7& 4& 1& 4\\ 3& 0& 2& 1\\ 1& 6& 8& 7\end{array}\right]\begin{array}{c}{\mathrm{v}}_1^{(1)}\\ {\mathrm{v}}_2^{(1)}\\ {\mathrm{v}}_3^{(1)}\\ {\mathrm{v}}_4^{(1)}\\ {\mathrm{v}}_5^{(1)}\end{array} \mathit{X}=\left[\begin{array}{cccc}4& 0& 0& 8\\ 3& 1& 9& 7\\ 0& 3& 1& 5\\ 2& 1& 4& 9\\ 6& 2& 8& 3\end{array}\right] \begin{array}{c}{\mathrm{X}}_1^{(1)}\\ {\mathrm{X}}_2^{(1)}\\ {\mathrm{X}}_3^{(1)}\\ {\mathrm{X}}_4^{(1)}\\ {\mathrm{X}}_5^{(1)}\end{array} \mathit{f}(\mathit{X})= \left[\begin{array}{c}80\\ 140\\ 35\\ 102\\ 113\end{array}\right]$$

$${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^1 = \left[\begin{array}{cccc}4& 0& 0& 8\\ 3& 1& 9& 7\\ 0& 3& 1& 5\\ 2& 1& 4& 9\\ 6& 2& 8& 3\end{array}\right]\begin{array}{c}{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 1}^{(1)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 2}^{(1)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 3}^{(1)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 4}^{(1)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 5}^{(1)}\end{array} {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = \left[\begin{array}{c}80\\ 140\\ 35\\ 102\\ 113\end{array}\right]$$

$${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$$

Iteration 2 (First solution):

To calculate ${\mathrm{v}}_1^{(2)}, {\mathrm{X}}_1^{(2)}$.

Let ${\mathrm{r}}_1 = \left[\begin{array}{cccc}0.4& 0.3& 0.9& 0.5\end{array}\right]$ and ${\mathrm{r}}_2 = \left[\begin{array}{cccc}0.8& 0.2& 0.7& 0.4\end{array}\right]$

$$\begin{array}{ll}{\mathrm{v}}_1^{(2)}& = \mathrm{w}{\mathrm{v}}_1^{(1)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 1}^{(1)} - {\mathrm{X}}_1^{(1)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_1^{(1)})\\ & = [1.5 5.1 1.75 3.8]\\ {\mathrm{X}}_1^{(2)}& = {\mathrm{X}}_1^{(1)} + {\mathrm{v}}_1^{(2)} \\ & = \left[4 0 0 8] + [1.5 5.1 1.75 3.8\right]\\ & =[5.5 5.1 1.75 11.8]\notin \left[0,10\right]\end{array}$$

This is the new position of first particle in the second iteration but new position is not within the bounds.

So, by using corner bounding strategy

$$\begin{array}{l}{\mathrm{X}}_1^{(2)} = \left[\begin{array}{cccc}5.5& 5.1& 1.75& 10\end{array}\right]\in \left[0,10\right]\\ \mathrm{f}({\mathrm{X}}_1^{(2)}) = 159.32\end{array}$$

No updation in ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 1}^{(2)} = 159.32\hspace{0.17em}> {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 1}^{(1)} = 80$ and

No updation in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} \therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} =\left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right]$ and ${\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$

Iteration 2 (Second Solution):

To Calculate ${\mathrm{v}}_2^{(2)}, {\mathrm{X}}_2^{(2)}$

$$\begin{array}{cc}{\mathrm{v}}_2^{(2)}& = \mathrm{w}{\mathrm{v}}_2^{(1)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 2}^{(1)} - {\mathrm{X}}_2^{(1)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_2^{(1)})\hfill \\ & = \left[\begin{array}{cccc}-0.1& 1.3& -6.3& -1.2\end{array}\right]\hfill \\ {\mathrm{X}}_2^{(2)}& = {\mathrm{X}}_2^{(1)} + {\mathrm{v}}_2^{(2)}\hfill \\ & =\left[3 1 9 7\right]+ \left[-0.1 1.3 -6.3 -1.2\right]\hfill \\ & =\left[2.9 2.3 2.7 5.8\right]\in \left[0,10\right]\hfill \\ \mathrm{f}({\mathrm{X}}_2^{(2)})\hfill & = 8.41\hspace{0.17em}+5.29\hspace{0.17em}+ 7.29\hspace{0.17em}+ 33.64\hspace{0.17em}= 54.63\hfill \end{array}$$

Update ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 2}^{(2)} = 54.63\hspace{0.17em}< {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 2}^{(1)} = 140$

No updation in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35 < {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 2}^{(2)} = 54.63$

$$\therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} =\left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$$

Iteration 2 (Third Solution):

$$\begin{array}{l}{\mathrm{v}}_3^{(2)} = \mathrm{w}{\mathrm{v}}_3^{(1)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 3}^{(1)} - {\mathrm{X}}_3^{(1)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_3^{(1)})\\ =\left[4.9 2.8 0.7 2.8\right]\end{array}$$

$$\begin{array}{l}{\mathrm{X}}_3^{(2)} = {\mathrm{X}}_3^{(1)} + {\mathrm{v}}_3^{(2)}= \left[0 \hspace{0.17em}3 \hspace{0.17em}1 \hspace{0.17em}5\right] + \left[4.9 \hspace{0.17em}2.8 \hspace{0.17em}0.7 \hspace{0.17em}2.8\right]\\ = \left[4.9 \hspace{0.17em}5.8 \hspace{0.17em}1.7 \hspace{0.17em}7.8\right]\in \left[0,10\right]\\ \mathrm{f}({\mathrm{X}}_3^{(2)}) =121.38\end{array}$$

No updation in ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 3}^{(2)} = 121.38\hspace{0.17em}> {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 3}^{(1)} = 35$

No updation in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, \therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} =\left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$

Iteration 2 (fourth Solution):

$$\begin{array}{ll}{\mathrm{v}}_4^{(2)}& = \mathrm{w}{\mathrm{v}}_4^{(1)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 4}^{(1)} - {\mathrm{X}}_4^{(1)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_4^{(1)})\\ & = \left[-0.3 0.6 -1.75 -1.7\right]\\ {\mathrm{X}}_4^{(2)}& = {\mathrm{X}}_4^{(1)} + {\mathrm{v}}_4^{(2)} = \left[2 \hspace{0.17em}1 \hspace{0.17em}4 \hspace{0.17em}9\right] + \left[-0.3 \hspace{0.17em}0.6 \hspace{0.17em}-1.75 \hspace{0.17em}-1.7\right]\\ & = \left[1.7 1.6 2.25 7.3\right]\in \left[0,10\right]\\ \mathrm{f}({\mathrm{X}}_4^{(2)})& = 2.89\hspace{0.17em}+ 2.56\hspace{0.17em}+ 5.0625\hspace{0.17em}+ 53.29\hspace{0.17em}= 63.8025\end{array}$$

Update ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 4}^{(2)} = 63.8025\hspace{0.17em} < {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 4}^{(1)} = 102$

No updation in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} \therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} =\left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$

Iteration 2 (Fifth Solution):

$$\begin{array}{ll}{\mathrm{v}}_5^{(2)}& = \mathrm{w}{\mathrm{v}}_5^{(1)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 5}^{(1)} - {\mathrm{X}}_5^{(1)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_5^{(1)})\\ & = \left[-6.5 4.5 -1.75 6.1\right]\end{array}$$

$$\begin{array}{l}{\mathrm{X}}_5^{(2)} = {\mathrm{X}}_5^{(1)} + {\mathrm{v}}_5^{(2)} \\ = \left[6 2 8 3\right] + \left[-6.5 4.5 -1.75 6.1\right]= \left[-0.5 6.5 6.25 9.1\right]\notin \left[0,10\right]\end{array}$$

$$\begin{array}{l}\therefore {\mathrm{X}}_5^{(2)} = \left[\begin{array}{cccc}0& 6.5& 6.25& 9.1\end{array}\right]\\ \mathrm{f}({\mathrm{X}}_5^{(2)}) = 42.25\hspace{0.17em}+ 39.0625\hspace{0.17em}+ 82.81\hspace{0.17em}= 164.1225\end{array}$$

No updation in ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 5}^{(2)} = 164.1225\hspace{0.17em} > {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 5}^{(1)} = 113$

No updation in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} \therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} =\left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$

So, in iteration 2, we get:

$$\mathit{V}=\left[\begin{array}{cccc}1.5& 5.1& 1.75& 3.8\\ -0.5& 1.3& -6.3& -1.2\\ 4.9& 2.8& 0.7& 2.8\\ -0.3& 0.6& -1.75& -1.7\\ -6.5& 4.5& -1.75& 6.1\end{array}\right]\begin{array}{c}{\mathrm{v}}_1^{(2)}\\ {\mathrm{v}}_2^{(2)}\\ {\mathrm{v}}_3^{(2)}\\ {\mathrm{v}}_4^{(2)}\\ {\mathrm{v}}_5^{(2)}\end{array} \mathit{X} = \left[\begin{array}{cccc}5.5& 5.1& 1.75& 10\\ 2.9& 2.3& 2.7& 5.8\\ 4.9& 5.8& 1.7& 7.8\\ 1.7& 1.6& 2.25& 7.3\\ 0& 6.5& 6.25& 9.1\end{array}\right]\begin{array}{c}{\mathrm{X}}_1^{(2)}\\ {\mathrm{X}}_2^{(2)}\\ {\mathrm{X}}_3^{(2)}\\ {\mathrm{X}}_4^{(2)}\\ {\mathrm{X}}_5^{(2)}\end{array} \mathit{f}(\mathit{X})= \left[\begin{array}{c}159.32\\ 54.63\\ 121.38\\ 63.8025\\ 164.1225\end{array}\right]$$

$${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{(2)} = \left[\begin{array}{cccc}4& 0& 0& 8\\ 2.9& 2.3& 2.7& 5.8\\ 0& 3& 1& 5\\ 1.7& 1.6& 2.25& 7.3\\ 6& 2& 8& 3\end{array}\right]\begin{array}{c}{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 1}^{(2)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 2}^{(2)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 3}^{(2)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 4}^{(2)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 5}^{(2)}\end{array} {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = \left[\begin{array}{c}80\\ 54.63\\ 35\\ 63.8025\\ 113\end{array}\right]$$

$${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right] {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$$

Iteration 3 (first Solution):

$$\begin{array}{l}{\mathrm{v}}_1^{(3)} = \mathrm{w}{\mathrm{v}}_1^{(2)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 1}^{(2)} - {\mathrm{X}}_1^{(2)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_1^{(2)})\\ = \left[-6.45 0.645 -1.925 1.16\right] \end{array}$$

$$\begin{array}{l}{\mathrm{X}}_1^{(3)} = {\mathrm{X}}_1^{(2)} + {\mathrm{v}}_1^{(3)} \\ = \left[5.5 5.1 1.75 10\right] + \left[-6.45 0.645 -1.925 1.16\right]\\ = \left[-0.95 5.745 -0.1725 11.16\right]\notin \left[0,10\right] \\ = \left[0 5.745 0 10\right]\\ \mathrm{f}({\mathrm{X}}_1^{(3)}) = 133.005025\end{array}$$

No update in ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} \mathrm{a}\mathrm{s} {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 1}^{(3)} = 133.005025\hspace{0.17em} > {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 1}^{(2)} = 80$

No update in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} \mathrm{a}\mathrm{s} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35 \therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} =\left[\begin{array}{cccc}0& 3& 1& 5\end{array}\right]$ and ${\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35$

Iteration 3 (second Solution):

$$\begin{array}{l}{\mathrm{v}}_2^{(3)} = \mathrm{w}{\mathrm{v}}_2^{(2)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 2}^{(2)} - {\mathrm{X}}_2^{(2)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_2^{(2)})\\ = \left[-3.55 1.12 -6.195 -1.32\right]\\ {\mathrm{X}}_2^{(3)} = {\mathrm{X}}_2^{(2)} + {\mathrm{v}}_2^{(3)} \\ = \left[-3.55 1.12 -6.195 -1.32\right] + \left[2.9 2.3 2.7 5.8\right]\end{array}$$

$$\begin{array}{l}= \left[-0.65 3.42 -3.495 4.48\right]\notin \left[0,10\right] \\ = \left[0 3.42 0 4.48\right]\\ \mathrm{f}({\mathrm{X}}_2^{(3)}) = 11.6964+ 20.0704\hspace{0.17em}= 31.7668\end{array}$$

Update ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 2}^{(3)} = 31.7668\hspace{0.17em} < {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 2}^{(2)} = 54.63$

Update ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35 > {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 2}^{(3)} = 31.7668\hspace{0.17em}$

$$\begin{array}{l}\\ \therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 3.42& 0& 4.48\end{array}\right] {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 31.7668\end{array}$$

Iteration 3 (third Solution):

$$\begin{array}{l}{\mathrm{v}}_3^{(3)} = \mathrm{w}{\mathrm{v}}_3^{(2)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 3}^{(2)} - {\mathrm{X}}_3^{(2)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_3^{(2)})\\ = \left[-5.39 -0.014 -2.24 -2.132\right]\\ {\mathrm{X}}_3^{(3)} = {\mathrm{X}}_3^{(2)} + {\mathrm{v}}_3^{(3)} \\ = \left[4.9 5.8 1.7 7.8\right] + \left[-5.39 -0.014 -2.24 -2.132\right]\\ =\left[-0.49 5.786 -0.54 5.668\right]\notin \left[0,10\right] \\ =\left[0 5.786 0 5.668\right]\\ \mathrm{f}({\mathrm{X}}_3^{(3)}) = 33.4778\hspace{0.17em}+ 32.1262\hspace{0.17em}=65.604024\end{array}$$

No updation in ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 3}^{(3)} = 65.604024\hspace{0.17em} > {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 3}^{(2)} = 35$

No Updation in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ $\therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 3.42& 0& 4.48\end{array}\right] \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 31.7668$

Iteration 3 (fourth Solution):

$$\begin{array}{l}{\mathrm{v}}_4^{(3)} = \mathrm{w}{\mathrm{v}}_4^{(2)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 4}^{(2)} -{\mathrm{X}}_4^{(2)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_4^{(2)})\\ = \left[-2.25 0.966 -3.5875 -2.882\right] \\ {\mathrm{X}}_4^{(3)} = {\mathrm{X}}_4^{(2)} + {\mathrm{v}}_4^{(3)} \\ =\left[1.7 1.6 2.25 7.3\right] + \left[-2.25 0.966 -3.5875 -2.882\right] \\ = \left[0 2.566 0 4.418\right]\\ \mathrm{f}({\mathrm{X}}_4^{(3)}) = 26.10308\end{array}$$

Update ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 4}^{(3)} = 26.10308\hspace{0.17em} < {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 4}^{(2)} = 63.8025$

Update ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 35 >{\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 4}^{(3)} = 26.10308$

$$\therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 2.566& 0& 4.418\end{array}\right] \mathrm{and} {\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 26.10308$$

Iteration 3 (fifth Solution):

$$\begin{array}{l}{\mathrm{v}}_5^{(3)} = \mathrm{w}{\mathrm{v}}_5^{(2)} + {\mathrm{c}}_1{\mathrm{r}}_1 ({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 5}^{(2)} - {\mathrm{X}}_5^{(2)}) + {\mathrm{c}}_2{\mathrm{r}}_2 ({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{} - {\mathrm{X}}_5^{(2)})\\ = \left[-0.95 -0.0552 -5.425 -3.1142\right] \\ {\mathrm{X}}_5^{(3)} = {\mathrm{X}}_5^{(2)} + {\mathrm{v}}_5^{(3)} \\ = \left[0 6.5 6.25 9.1\right] + \left[-0.95 -0.0552 -5.425 -3.1142\right]\\ = \left[0 6.4448 0.825 5.9858\right]\\ \mathrm{f}({\mathrm{X}}_5^{(3)}) = 78.045825\end{array}$$

Update ${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$ as ${\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 5}^{(3)} = 78.045825\hspace{0.17em} < {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}, 5}^{(2)} = 113$

No update in ${\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}$

$$\therefore {\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}} = \left[\begin{array}{cccc}0& 2.566& 0& 4.418\end{array}\right]\mathrm{and}{\mathrm{f}}_{{\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = 26.10308$$

So, in iteration 3:

$$\mathrm{V}= \left[\begin{array}{cccc}-6.45& 0.645& -1.925& 1.16\\ -3.55& 1.12& -6.195& -1.32\\ -5.39& -0.014& -2.24& -2.132\\ -2.25& 0.966& -3.5875& -2.882\\ -0.95& -0.0552& -5.425& -3.1142\end{array}\right]\begin{array}{c}{\mathrm{v}}_1^{(3)}\\ {\mathrm{v}}_2^{(3)}\\ {\mathrm{v}}_3^{(3)}\\ {\mathrm{v}}_4^{(3)}\\ {\mathrm{v}}_5^{(3)}\end{array},$$

$$\mathrm{X}=\left[\begin{array}{cccc}0& 5.745& 0& 10\\ 0& 3.42& 0& 4.48\\ 0& 5.786& 0& 5.668\\ 0& 2.566& 0& 4.418\\ 0& 6.4448& 0.825& 5.9858\end{array}\right]\begin{array}{c}{\mathrm{X}}_1^{(3)}\\ {\mathrm{X}}_2^{(3)}\\ {\mathrm{X}}_3^{(3)}\\ {\mathrm{X}}_4^{(3)}\\ {\mathrm{X}}_5^{(3)}\end{array} \mathrm{f}(\mathrm{X})= \left[\begin{array}{c}133.005\\ 31.7668\\ 65.604024\\ 26.10308\\ 78.046\end{array}\right]$$

$${\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}^{(2)} = \left[\begin{array}{cccc}4& 0& 0& 8\\ 0& 3.42& 0& 4.48\\ 0& 3& 1& 5\\ 0& 2.566& 0& 4.418\\ 0& 6.4448& 0.825& 5.9858\end{array}\right]\begin{array}{c}{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 1}^{(3)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 2}^{(3)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 3}^{(3)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 4}^{(3)}\\ {\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}, 5}^{(3)}\end{array} {\mathrm{f}}_{{\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}} = \left[\begin{array}{c}80\\ 31.7668\\ 35\\ 26.103088\\ 78.046\end{array}\right]$$

Similarly, other iterations can be performed. This example is solved completely up to the third iteration only. This is the just the explanation of the working of particle swarm optimization (PSO) algorithm through an example. It is not possible to obtain an optimal solution by manual calculations. Further iterations can be performed through proper software for the PSO algorithm.

3. Parameters of PSO

There are a number of control parameters in the particle swarm optimization (PSO) algorithm, namely the number of particles, size of neighborhood, number of iterations, acceleration coefficients, inertia weight, dimension of the problem and random values associated with the cognitive and social component of the velocity update equation. In addition, if velocity clamping and constriction are used, then maximum velocity and coefficient of constriction are also considered to be very effective control parameters (Engelbrecht, A.P.) [15]. These parameters have a great impact on the convergence speed, performance and quality of the solution obtained. The fine tuning of the parameters is required, to avoid premature convergence. Discussions of these parameters are as below:

3.1. Population Size

Swarm size (n_s) is the number of particles in the whole swarm. A large number of the particles cover a larger part of the search space per iteration, but it also increases the computational complexity per iteration. It can also be possible that a larger number of particles may lead to a lesser number of iterations to obtain an optimal solution. From the heuristics found in the publications, the success rate of getting a good solution in the PSO algorithm is greater with a swarm size of 20 to 50 particles (Eberhart and Shi) [16]. However, it is also found that the optimal population size is usually problem dependent.

3.2. Number of Iterations

In the same way as the swarm size, the number of iterations is also problem dependent, to obtain a good solution in the particle swarm optimization (PSO) algorithm. On the one hand, a lesser number of iterations may prematurely stop the search process, and on the other hand, a larger number of iterations only adds to the computational complexity in obtaining a quality solution, if the termination criterion depends on a number of iterations.

3.3. Neighborhood Size

The size of the neighborhood is related to the degree of social interaction among the swarm members. The less interaction that there is in a neighborhood of a smaller size leads to a slow but reliable convergence. The more interaction that there is in a neighborhood of a larger size leads to a fast convergence. For instance, if the size of a neighborhood is defined as two, then particle (k) compares its fitness value with particle (k − 1) and particle (k + 1) (Eberhart and Shi) [16]. From the heuristics found in the publications, the size of a neighborhood is usually taken to be 15% of the population size in most of the applications.

3.4. Acceleration Coefficients

The impact of the cognitive and social component on the particle’s velocity is managed by the acceleration coefficients ${\mathrm{c}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{c}}_2$, along with the random numbers, ${\mathrm{r}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{r}}_2$. A proper balance between the values of ${\mathrm{c}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{c}}_2$ is required, as inappropriate values of ${\mathrm{c}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{c}}_2$ may result in divergent and cyclic behavior of the whole swarm. Impact of different values of ${\mathrm{c}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{c}}_2$ is explained in

Table 3. Impact of different values of Acceleration Coefficients.

Values of Acceleration Coefficients	Impact of Acceleration Coefficients	Corresponding Change in Velocity Update Equation (1)
c1 = c2 = 0	In this case, the particle moves on with the same current speed till it reaches the boundary of the search space as its velocity is independent from the impact of personal best and global best position.	${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1) = {\mathrm{V}}_{\mathrm{k}}(\mathrm{i} )$
c1 > 0 and c2 = 0	Here, the social component of the velocity does not influence the particle’s velocity and particle will move in the global search space according to its own best position.	$\begin{array}{l}{\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1) = {\mathrm{V}}_{\mathrm{k}}(\mathrm{i} ) + \\ {\mathrm{c}}_{1 }{\mathrm{r}}_1\left({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{\mathrm{k}} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right)\end{array}$
c1 = 0 and c2 > 0	Here, the cognitive component of the velocity does not influence the particle’s velocity and particle will move in the global search space according to its neighbor’s best position.	$\begin{array}{l}{\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1) = {\mathrm{V}}_{\mathrm{k}}(\mathrm{i} ) + \\ {\mathrm{c}}_{2 }{\mathrm{r}}_2\left({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right)\end{array}$
c1 = c2	The particle is attracted towards the average of both pbest and gbest position.
c1 >> c2	Here, the personal best position will generally effect the particle’s velocity more, that leads to excessive wandering in the search space.
c1 << c2	Here, the best position of other members of the swarm has more impact on particle’s velocity that leads to pre mature convergence.

.

3.5. Velocity Clamping

It is the property of a good optimization algorithm to create an optimal balance between the exploration and exploitation objectives. It was found in the basic PSO algorithm that the velocity quickly explodes to larger values and, as a result of it, the position of the particle changes rapidly, that leads to the divergence of the whole swarm. To overcome this problem, Eberhart and Kennedy [16] introduced the concept of velocity clamping i.e., the velocities are clamped to stay within the boundary constraints. V_max denotes the maximum allowed velocity that creates a better equilibrium between the global exploration and local exploitation. If the velocity of the particle reaches beyond this specified limit V_max, then that velocity of the particle is set to the maximum velocity value (V_max). V_max is an important parameter as it controls the explosion of velocity. Higher values of V_max encourage global exploration, but it may risk the possibility that the particles may miss a better search area. Since the particles are moving faster, they may jump over the optimal solution and continue to roam in a useless region in the global search area, and on the other hand, the smaller values of V_max encourage local exploitation but may risk the possibility that the particles become trapped in the local optima, resulting in the particles not exploring a better search space. So, to create a balance between exploration and exploitation, an appropriate value of V_max is required.

We can write Vmax as written in Equation (4):

V_max = δ (X_max − X_min)

(4)

where X_max and X_min are, respectively, the maximum and minimum values of X and δ ε (0, 1].

3.6. Constriction Coefficient

To ensure and enhance the convergence speed of the PSO algorithm and to ensure a balance between exploration and exploitation, Maurice Clerc [17] suggested the parameter constriction coefficient $\chi$ (Clerc and Kennedy) [18]. He implemented it on the basic velocity update Equation (1) of the original PSO algorithm.

The velocity update Equation (1) changes to Equation (5):

$${\mathrm{V}}_{\mathrm{k}}(\mathrm{i} + 1) = \chi [{\mathrm{V}}_{\mathrm{k}}(\mathrm{i} ) + {\mathrm{c}}_{1 }{\mathrm{r}}_1\left({\mathrm{p}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{\mathrm{k}} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right) + {\mathrm{c}}_{2 }{\mathrm{r}}_2\left({\mathrm{g}}_{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t},\mathrm{i}}^{} - {\mathrm{X}}_{\mathrm{k}}(\mathrm{i})\right)]$$

(5)

where the constriction coefficient $\chi$ is given by

$$\chi = \frac{2\mathsf{\kappa }}{\left|2 - \mathsf{\phi } - \sqrt{\mathsf{\phi } \left(\mathsf{\phi } - 4\right)}\right|}$$

(6)

where $\mathsf{\phi }= {\mathsf{\phi }}_1 + {\mathsf{\phi }}_2$ and ${\mathsf{\phi }}_{1 } = {\mathrm{c}}_1 {\mathrm{r}}_1,{\mathsf{\phi }}_{2 } = {\mathrm{c}}_2 {\mathrm{r}}_2$, $\mathsf{\kappa }$ is a function of ${\mathrm{c}}_1$ and ${\mathrm{c}}_2$.

Equation (6) is used under the condition that $\mathsf{\phi }\ge 4$ and $\mathsf{\kappa } \in \left[0,1\right]$.

Parameter $\mathsf{\kappa }$ controls the exploitation and exploration abilities of the swarm. If $\mathsf{\kappa }$ ≈ 0 then the convergence is faster with local exploitation, and if $\mathsf{\kappa }$ ≈ 1 then the convergence is slow with a higher degree of exploration. Usually $\mathsf{\kappa }$ is set to a constant value. However, a higher degree of exploration with local exploitation can be achieved by taking the initial value of $\mathsf{\kappa }$ near to 1, then decreasing it to 0. When Clerc’s constriction factor is used with the PSO algorithm, then $\mathsf{\phi } = 4.1\hspace{0.17em}\mathrm{a}\mathrm{n}\mathrm{d} \mathrm{t}\mathrm{h}\mathrm{u}\mathrm{s} \mathrm{c}o\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t} \mathrm{m}\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{i}\mathrm{p}\mathrm{l}\mathrm{i}\mathrm{e}\mathrm{r} \mathsf{\kappa } \approx 0.729.$

3.7. Inertia Weight

The notion of inertia weight ‘ω’ of a particle in the basic PSO algorithm was first introduced into the literature by Shi and Eberhart [14], to improve its convergence performance. This coefficient controls the global and local exploration and the exploitation capacity of the swarm, by determining the influence of the previous velocity on the particle’s current movement. With the introduction of this new parameter ω, the velocity update Equation (1) is changed to velocity update Equation (3). The suitable value of ω results in fewer iterations on average to find a sufficiently optimal solution. To balance the global and local search and to ensure the convergence to a sufficiently optimal solution in a lesser number of iterations, an adjustment in the value of the inertia weight ω, is extremely important. Larger values of ω facilitate the global exploration and smaller values promote the local exploitation. As originally proposed, the value of the inertia weight ω is often decreased linearly from about 0.9 (ω_max) to 0.4 (ω_min) during a run.

It is described as follows Equation (7) (Eberhart and Shi [19]):

$$\omega = {\omega }_{\mathrm{m}\mathrm{a}\mathrm{x}} - \frac{{\omega }_{\mathrm{m}\mathrm{a}\mathrm{x}}- {\omega }_{\mathrm{m}\mathrm{i}\mathrm{n}}}{\mathrm{i}\mathrm{t}\mathrm{e}{\mathrm{r}}_{\mathrm{m}\mathrm{a}\mathrm{x}}}\times \mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}$$

(7)

where $\mathrm{i}\mathrm{t}\mathrm{e}{\mathrm{r}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the maximum number of iterations, iter is the current iteration, ${\omega }_{\mathrm{m}\mathrm{a}\mathrm{x}}$ and ${\omega }_{\mathrm{m}\mathrm{i}\mathrm{n}}$ are initial and final values of the inertia weight respectively.

Van den Bergh and Engelbrecht [20] stated a strong relationship ( see Equation (8)) between acceleration coefficients c₁, c₂ and inertia weight ω as:

$$\omega > \frac{1}{2} ( {\mathrm{c}}_1 + {\mathrm{c}}_2) - 1$$

(8)

Eberhart and Shi [19] made a comparison between the constriction coefficient ($\chi$) and inertia weight ($\mathsf{\omega }$). The authors considered that both of the approaches are equally effective. As both the constriction coefficient and inertia weight aim to create a balance between the exploration and exploitation capabilities, this results in an improvement in the convergence time and quality of solution obtained. Low values of the constriction coefficient and inertia weight result in exploitation, whereas higher values result in exploration. Eberhart and Shi [19] also proved that if velocity clamping and constriction coefficient are used together, even then convergence can be faster.

4. Advances on Particle Swarm Optimization (PSO) Algorithm

Due to the simplicity and wide range of applications of the particle swarm optimization (PSO) algorithm, it has gained more attention from a variety of researchers belonging to different fields. Houssein et al. [21] presented a rigorous and systematic review on the developments, recent trends, hybridization, parallelization and diverse applications of the particle swarm optimization (PSO) algorithm. Many researchers are working on the advancement of the PSO algorithm. As a result, more research is being completed in this area, that has led several researchers to report problems with the original PSO algorithm, such as premature convergence, performance issues, etc. In order to lessen these types of problems and improve its effectiveness and efficiency, researchers have made many advances on the original PSO algorithm. The advances on PSO algorithm have been sectioned into four categories:

• Modifications of original PSO;
• Extensions of applications of PSO;
• Theoretical analysis on PSO;
• Hybridization of PSO.

4.1. Modifications of Original PSO

There are modifications of the PSO algorithm, including fuzzy PSO, bare-bones PSO, etc. Krohling and Renato [22] made a modification in the original particle swarm algorithm by introducing the concept of Gaussian probability distribution. In their proposed algorithm, they only needed the parameter ‘the number of particles’ to be specified before actually using the algorithm. The experimental results of the Gaussian particle swarm algorithm showed that the Gaussian swarm outperformed the standard PSO. Baskar and Suganthan [23] introduced a new version of the PSO, called concurrent PSO (CONPSO), to improve the convergence performance of the original PSO. By testing this new version of the PSO on six classical functions, the authors proved that this new version of the PSO, i.e., the CONPSO, outperformed many other approaches of PSO. Kennedy and Eberhart [24] reworked the original particle warm optimization (PSO) technique and introduced a binary particle swarm. According to the authors, this new version of the PSO algorithm could optimize any function, continuous or discrete. They discussed this algorithm with various examples and applications. To make the basic particle swarm algorithm more understandable, Kennedy [25] eliminated the velocity formula. The author compared several variations and discovered its similarity with other stochastic population-based problem-solving methods, and suggested new avenues of investigation.

Mendes et al. [26] modified the original PSO algorithm by the concept of fully informed particles in the original PSO, as they felt that each of the particles in the swarm need not be affected only by the best performer among his neighbors. With the help of numerical experiments, the authors revealed that this new modified version of PSO outperformed the original PSO. Cervantes et al. [27] proposed to resolve the classification problems by making some modifications in the original PSO. So, a binary version of the PSO algorithm was compared with some machine-learning techniques, and the authors showed the promising results. Shi and Eberhart [28] made use of a fuzzy system to adopt the inertia weight of the particle swarm optimization (PSO) algorithm and introduced a new variant of PSO, called fuzzy adaptive PSO. Experiments on three test functions concluded that this fuzzy adaptive PSO was a promising optimization technique.

To detect facial emotion, Ghandi et al. [29] presented a novel approach called guided particle s optimization (GPSO) by modifying the original particle swarm optimization (PSO) algorithm. The authors proved that their emotion detection algorithm gave promising results in terms of accuracy in the detection of emotions. In order to obtain an optimum solution, Tanweer et al. [30] derived a new PSO algorithm, namely the self-regulating PSO (SRPSO) algorithm, by incorporating the best of the human-learning strategies. The authors proposed the learning strategies, on the one hand using a self-regulatory inertia weight for better exploration, and on the other hand using self-perception of the global search space for better exploitation. The statistical analyses indicated the superiority of the SRPSO over other metaheuristics. Presently, many researchers are also working on the further modification of the original PSO algorithm.

4.2. Extensions of Applications of PSO

Since its origin in 1995, researchers have also widened/extended the range of application of the original PSO algorithm to a variety of optimization problems, such as constrained, multi objective, multi modal, discrete and binary optimization. To solve the multi-objective optimization problems, Seok et al. [31] introduced a modified variant of the PSO algorithm, namely the homogeneous particle swarm optimizer (HPSO). The authors proved the excellent exploration ability of the proposed algorithm with other multi-objective PSO algorithms by making a comparison study. The authors, Coello and Lechuga [32], extended the application of the original particle swarm optimization (PSO) to solve the multi-objective optimization problems by using a Pareto dominance approach to determine the flight direction of the particle. By testing it with some standard test functions, the authors concluded that their approach was highly competitive in comparison to the other multi-objective optimization techniques. Mohamed et al. [33] proposed a new algorithm by hybridizing particle swarm optimization (PSO) with two other statistical techniques—Ranking and Selection (R&S), and Mean Square Error Criterion (MSE), in order to solve the stochastic optimization problems, called (STPSO). The authors proved the robustness of their proposed algorithm with experimental results. In order to solve the NP hard knapsack problems (KPs), Bansal and Deep [34] designed a modified binary particle swarm optimization (MBPSO) algorithm. The authors tested this new modified binary PSO version on 10 benchmark problems, and by comparing the results with both the binary particle swarm optimization (BPSO) and the genotype phenotype-modified binary particle swarm optimization (GPMBPSO), showed its effectiveness in terms of reliability, cost and the quality of the solution obtained over the other algorithms.

To efficiently solve the different problems on power systems, Rahmani et al. [35] proposed an evolutionary modified particle swarm optimization. On the basis of numerical experiments, the authors proved that their proposed algorithm was efficiently able to solve the constrained economic dispatch (ED) problem with a comparatively higher rate of convergence than the basic PSO. To efficiently solve the clustering problem, Neshat and Yazdi [36] proposed a new cooperative algorithm that utilized the global search ability of the PSO algorithm and the local search ability of k-means. The authors made a comparison of their proposed algorithm with other algorithms, such as PSO with constriction factor (CF-PSO) and k-means algorithms and revealed the convergence efficiency of their proposed algorithm. In order to solve the university course timetabling problems using PSO, the authors, Kanoh and Chen [37], introduced a new algorithm, using transition probability into the basic PSO algorithm. With the help of the experimental results, the authors demonstrated the effectiveness of their proposed algorithm over evolutionary strategy to solve the time-table problems. Garsva and Danenas [38] presented a new PSO algorithm by using a linear SVM (support vector machine) approach to solve both small and large scale classification problems. On the basis of the experiments, the authors concluded that their proposed PSO-LinSVM approach gave comparatively better results than other similar approaches.

The basic PSO algorithm does not work well for solving problems of mobile robot path planning, due to its slow convergence and limitations on application. To overcome this, the authors, Li et al. [39], derived the improved particle swarm optimization (IPSO) algorithm. By making a comparative study with other approaches on the standard benchmark functions, the authors proved that their derived algorithm gave a better solution with a lower number of iterations. In order to monitor the conditions of the surge arrester, Hoang et al. [40] proposed a novel differential particle swarm optimization(PSO)-based support vector machine (SVM) classifier, namely (DPSO-SVM). The authors, through experiments on several benchmark functions, showed the superiority of the proposed algorithm (DPSO-SVM) in order to improvise the classifier’s performance. In neuro fuzzy system (NFS), usually a gradient-based algorithm was used by the researchers, but Karakuzu et al. [41] used an improved PSO (iPSO) to introduce the first embedded high-speed, low-cost implementation of the neuro fuzzy network (NFN) hardware through online training on a field programmable gate array (FPGA). The authors tested its proposed implementation method on practical problems. The results showed the efficiency of the proposed implementation over other approaches available in the literature. In order to solve a power consumption minimization problem, Liao et al. [42] suggested a PSO-based algorithm, namely distribution PSO (DPSO). The improved algorithm of PSO was checked on a luminous control problem and the results showed the effectiveness of the modified algorithm, along with its suitability to be parallelized. The same optimization problem performed differently with different settings of the parameters. Based on the concept of the building block thesis, Li et al. [43] proposed two variants to the PSO algorithm, namely the PSO with a fixed phase (PSOFP) algorithm, and the PSO with a dynamic phase (PSODP) algorithm. These new variants were tested on benchmark functions to solve the single objective numerical optimization, and revealed the robustness of the proposed variant of the PSO algorithm.

The areas of application for the PSO algorithm has been extended by many researchers, but there is still a need to explore more areas of application of the PSO algorithm.

4.3. Theoretical Analysis on PSO

By realizing the importance of the appropriate values of the control parameters in the PSO algorithm, such as the inertia weight, acceleration coefficients, number of particles, etc., the researchers diverted their attention towards the theoretical aspects of the PSO algorithm and derived many theories related to parametric selection to obtain an optimal solution in a lower number of iterations and with a higher rate of convergence. Their theoretical analysis includes parametric setting, convergence analysis, etc. Shi and Eberhart [14] had proposed a new PSO parameter, called inertia weight ‘$\omega$’ into the original particle swarm optimizer to improve its convergence performance. The authors explained the impact of this new parameter through examples. Shi and Eberhart [44] investigated the convergence performance of the particle swarm optimization (PSO) algorithm with a linearly decreasing inertia weight, and tested it on four non-linear functions. On the basis of the results, the authors concluded that some of the disadvantages of PSO could be overcome by adjusting the inertia weight. By obtaining motivation from the Gaussian and Cauchy probability distribution in the PSO algorithm, the authors Krohling and Coelho [45] proposed a modified variant of the PSO, namely PSO-E, that made use of the exponential probability distribution to upgrade the convergence performance. The numerical results on well-known classical functions indicated the competence of PSO-E in finding the solution in a lower number of iterations. Feng et al. [46] proposed a new adaptive inertia weight approach in the particle swarm optimization (PSP) algorithm. The performance of this newly proposed variant of PSO was checked on three classical benchmark functions and the results were also analyzed to prove its higher speed of convergence and more capability in locating the optimal solution in comparison to other optimization techniques. In order to balance between the local exploitation and the global exploration abilities for the PSO variant, Qin et al. [47] introduced a modified variant of PSO, called the adaptive inertia weight PSO algorithm (AIW-PSO). The authors proved the efficiency of the proposed algorithm (AIW-PSO) by comparing its results with other modified variants of the PSO, namely fuzzy adaptive inertia weight PSO, random number inertia weight PSO and linearly decreasing inertia weight PSO. The results showed that this new algorithm, AIW-PSO, outperformed the other algorithms.

Keeping in mind the importance of the parameter inertia weight in the PSO, Jiao et al. [48] proposed a dynamic inertia weight particle swarm optimization algorithm. The authors tested their improved PSO on six classical problems and concluded that the proposed variant improved the search performance of the standard particle swarm optimization (SPSO) algorithm.

Inertia weight is considered to be the most important parameter in the PSO algorithm. Usually, ω is chosen to be of small value in the range [0, 1] but Deep et al. [49] considered varying the values of ω, and proposed two variants of PSO, namely the globally adaptive inertia weight (GAIW) PSO and the locally adaptive inertia weight (LAIW) PSO. By testing these variants on various benchmark problems, the authors concluded that the proposed variant outperformed the basic PSO in terms of accuracy, efficiency and convergence speed. To improvise the convergence speed and to overcome the stagnation behavior of the basic PSO algorithm, Deep et al. [50] proposed a new variant of the basic PSO called fine grained inertia weight PSO (FGIWPSO). By testing their proposed variant of PSO on ten benchmark functions, the authors showed its effectiveness over the basic PSO, in terms of the quality of the solution obtained and the rate of convergence. To overcome the disadvantages of premature convergence of the PSO, Isiet and Gadala [51] conducted an extensive sensitivity analysis on the impact of the control parameters of the PSO algorithm. For parametric analysis, the authors considered a constraint optimization problem, and discovered that the PSO is most sensitive to inertia weight (ω) and acceleration coefficients (c₁ and c₂), and therefore suggested some optimal parametric combinations, and on the basis of a verification study of the suggested parameters showed that the proposed PSO could outperform some other metaheuristics.

4.4. Hybridization of PSO Algorithm

When the PSO algorithm is combined with some traditional and evolutionary optimization algorithms, in order to improve the effectiveness and efficiency of the algorithm and to compensate for the weakness of each other, then that is termed as the hybridization of the PSO. One can also say that combining the advantages of the PSO with the advantages of another optimization algorithm is called the hybridization of PSO algorithm, or the blending of the PSO with another optimization algorithm. The researchers hybridized PSO with many other algorithms, including the genetic algorithm (GA), the differential evolution (DE), the Ant Colony optimization (ACO), the simulated annealing (SA), the artificial Bee colony (ABC), etc. and found very promising results.

Nowadays, the PSO has become one of the researchers’ choices for the purposes of hybridization. Deep and Bansal [52] introduced the qPSO algorithm by hybridizing the PSO with a quadratic approximation operator. The authors considered the two variants of the PSO algorithm, namely the PSO with linearly decreasing inertia (PSO-W) and the PSO with constriction factor (PSO-C) for the purpose of hybridization, and proposed two new variant of PSO, namely qPSO-W and qPSO-C. The performance of the proposed variant was tested on 15 benchmark problems. The numerical results showed that the new hybrid PSO variants outperformed the basic PSO variants. For the purpose of microgrid optimization, Bao et al. [53] compared their proposed Multi-PSO-SVM prediction model with the other three algorithms and proved its superiority over them. To force the PSO algorithm to jump out of the stagnation situation, Liua et al. [54] hybridized particle swarm optimization (PSO) with differential evolution (DE) and proposed a novel hybrid algorithm, namely PSO-DE. On the basis of numerical experiments on constrained numerical and engineering optimization functions, the authors showed that their proposed novel approach resolved the convergence issues of basic PSO and also improved the quality of the solution obtained.

To overcome the various problems related to the PSO algorithm, Pu et al. [55] proposed an HPSO algorithm, and with experimental results proved the higher rate of stability, clustering efficiency and the global search ability of their algorithm over others. By using python tools, Mao et al. [56] developed a PSO-LSTM-based model to investigate the speed of road traffic in Kunming City, China. Singh et al. [57] introduced a new hybrid version of the particle swarm optimization (PSO) algorithm, namely the hybrid particle swarm optimization (HPSO) by combining two different approaches of PSO i.e., standard particle swarm optimization (SPSO) and mean particle swarm optimization (MPSO). By conducting numerical experiments on several benchmark problems, the authors concluded that that their proposed algorithm worked efficiently in comparison to SPSO and MPSO, in terms of speed and quality of solution. For the purpose of grid scheduling and resource management, Ankita and Sahana [58] proposed a PSO-based Ba-PSO scheduling algorithm. Song et al. [59] introduced a PSO-SVM regression model and proved its higher accuracy by proving that the average relative errors of the regression results of the specific supported roadway i.e., the wood-supported roadway, the I-steel-supported roadway and the bolt-net-supported roadway were all <5%.

Yang [60] proposed an improved PSO-BP combined forecasting model. This newly combined forecasting model is used in the college student entrepreneurship prediction experiment. Pozna et al. [61] presented a hybrid version of PSO, namely Particle Filter-Particle Swarm Optimization (PF-PSO) algorithm and proved its application to the fuzzy-controlled servo system. To obtain an optimal solution for the dynamic facility layout problem (DFLP), Nasab and Emami [62] proposed a hybrid PSO (HPSO) that mapped the discrete feasible space of the DFL problem into a continuous space with the help of coding procedure, and after that, the search for good quality solutions of the problem. For further improvements in their proposed algorithm, the authors combined it with simulated annealing. With the help of different experiments using their proposed algorithm, they proved that it was comparatively efficient compared to other variants. To create a balance between the global and local search abilities, Esmin and Matwin [63] proposed a hybrid PSO algorithm (HPSOM) by integrating PSO with a genetic algorithm, using the mutation process. On the basis of the numerical experiment, the authors showed that their proposed method significantly outperformed SPSO in terms of factors such as convergence speed, stability and the quality of the solution obtained.

In order to stabilize the stability of the power system, Elazim and Ali [64] hybridized the particle swarm optimization (PSO) and the bacterial foraging optimization algorithm (BFOA), and proposed a new version, namely the bacterial swarm optimization (BSO). To check the validity of the proposed version BSO, numerical experiments were conducted by the authors and the results presented the effectiveness of the proposed controller over the original PSO and BFOA algorithm. In order to optimize the learning rate of the neural network, Enireddy and Kumar [65] combined the cuckoo search algorithm with the particle swarm optimization (PSO) algorithm and proposed an improved image classification algorithm for content-based image retrieval (CBIR). On the basis of the experimental results, the authors showed the classification accuracy and faster learning with this proposed neural network algorithm.

Garg [66] proposed a guided hybrid approach, namely a PSO-GA algorithm, in order to solve the constrained optimization problems. On the basis of a comparative study conducted on various engineering design optimization problems, the author proved the effectiveness of the proposed algorithm over other evolutionary algorithms in searching for the near optimal global solution. Singh and Singh [67] proposed a new hybrid variant of particle swarm optimization (PSO) and the Grey Wolf optimizer (GWO), namely the HPSOGWO, by using exploitation capabilities from PSO and exploration capabilities from GWO. The authors tested their newly derived hybrid variant on 23 classical problems and by their experimental results showed that the newly developed variant outperformed other algorithms in respect of convergence and the accuracy of the solution obtained. Al-Thanoon et al. [68] presented a new hybrid algorithm by combining the advantages and strengths of both the particle swarm optimization (PSO) algorithm and the firefly algorithm (FFA), to provide a balance between the exploration and exploitation capabilities. The authors compared the performance of the newly proposed algorithm with the standard particle swarm optimization (PSO) algorithm and firefly algorithm (FFA) and yielded better results by proving its efficiency in obtaining a high classification performance.

To solve different nonlinear and convex optimal power flow (OPF) problems, Khan et al. [69] introduced an effective and novel hybrid firefly particle swarm optimization (HFPSO) algorithm. They coded their technique, using MATLAB software, and checked its effectiveness on several test functions, and showed that the proposed algorithm had a faster rate of convergence and had more capability to handle several complex OPF problems. Qinghai [70] explained the basic principles of the PSO algorithm, and discussed its advantages and disadvantages. He discussed some of the improved and hybrid versions of the standard particle swarm optimization (SPSO) algorithm and suggested its future research scope. Nowadays, researchers are working more in the area of hybridization, because hybridization makes it possible to combine the advantages of the hybridized algorithms by setting aside their weaknesses, leading to the development of a highly efficient hybridized algorithm.

5. Conclusions

On the one hand particle swarm optimization (PSO) algorithm is popular among the scientific community, due to its simplicity, easy implementation and competence to be applied to a wide range of problems. On the other hand, the modifications, extensions and hybridization of the PSO algorithm during the last few decades indicates that there are a lot more opportunities to make improvements to the algorithm, to increase the effectiveness of the results. The researchers have worked on the limitations of the PSO algorithm, such as obtaining trapped in the local optima, premature convergence, impropriate accuracy, etc. The researchers have tried to make it more applicable to more classes of optimization problems. In the present paper, by presenting an overview of the PSO algorithm, we have explained the basic concepts and parameters of PSO, along with a variety of the advancements of the PSO algorithm. From the current literature survey on the PSO algorithm, we have also found that a large amount of research has been carried out on the PSO algorithm, but more areas of the applications of PSO can be enhanced in the future research. Research both on the applications aspect and the hybridization of the algorithm can also be conducted. The theoretical aspect can also be further explored in future research, in order to lessen the number of known parameters and criteria of their selection for the easy implementation of the particle swarm optimization (PSO) algorithm to its wide range of applications.