An Improved Particle Swarm Optimization Algorithm for Data Classification

Abstract

Optimisation-based methods are enormously used in the field of data classification. Particle Swarm Optimization (PSO) is a metaheuristic algorithm based on swarm intelligence, widely used to solve global optimisation problems throughout the real world. The main problem PSO faces is premature convergence due to lack of diversity, and it is usually stuck in local minima when dealing with complex real-world problems. In meta-heuristic algorithms, population initialisation is an important factor affecting population diversity and convergence speed. In this study, we propose an improved PSO algorithm variant that enhances convergence speed and population diversity by applying pseudo-random sequences and opposite rank inertia weights instead of using random distributions for initialisation. This paper also presents a novel initialisation population method using a quasi-random sequence (Faure) to create the initialisation of the swarm, and through the opposition-based method, an opposite swarm is generated. We proposed an opposition rank-based inertia weight approach to adjust the inertia weights of particles to increase the performance of the standard PSO. The proposed algorithm (ORIW-PSO-F) has been tested to optimise the weight of the feed-forward neural network for fifteen data sets taken from UCI. The proposed techniques’ experiment result depicts much better performance than other existing techniques.

1. Introduction

Data classification is widely used in machine learning to solve problems such as spam email filtering, social network analysis, biological data analysis, diagnosing medical diseases, image and speech recognition [1]. The classification process contains two steps: the first step is training, which builds a model from the training samples, and the second step is the model predicting the labels of unlabeled test samples. In the machine learning field, an artificial neural network (ANN) is a classification problem-solving technique [2]. ANNs trained with Back-Propagation (BP) based on gradient descent technique generally slow convergence, are stuck in local optima, and require more training time [3]. The use of evolutionary algorithms in ANN teaching overcomes these shortcomings.

Swarm Intelligence is a field that consists of many individuals and deals with natural and artificial systems [4]. These systems are coordinated through self-organisation and decentralised control. In particular, this area focuses on collective behaviour that is displayed as a result of regional interactions between people and the environment [5]. Examples of checking systems through swarm intelligence are ants and termite swarms, fish swarms, bird swarms, and terrestrial animal swarms [6]. The swarm intelligence population consists of simple agents that are closer to the optimal outcome and interact with each other and environments. Emerging intelligence that communicates with its ecosystem is based on simple independent agents, is composed of other agents but behaves relatively individually from all other agents. Independent agents do not follow global plans or the instruction of other participants. Over the past two decades, swarm intelligence and nature-inspired computing have generated a lot of interest in almost all fields of science, industry, and engineering. Some human artifacts also belong to the field of swarm intelligence; in particular, some multi-robot systems. They are written to solve data analysis [7] and optimisation problems. Particle Swarm Optimization (PSO) [8], Cat Swarm Optimization (CSO) [9], Artificial Bee Colony (ABC) [10], Cuckoo Search (CS) [11] and Bat algorithm (BA) [12] belong to SI.

Particle Swarm Optimization (PSO) is a well-known example of swarm intelligence, introduced by Kennedy and Eberhart in 1995 [13] to solve global optimisation problems. Because of its simplicity and efficiency, it has been described in various engineering fields and has become the most effective method for solving optimisation problems. In PSO, different numbers of particles are known as a swarm, which search for the best possible solutions in search space [13,14,15]. In PSO, individuals are known as particles searching from one position to another; if any particle finds food without wasting time getting the food and sharing information of its position with another particle, all particles have to follow to reach that position [16]. Each particle follows the basic rule for determining its previous best position or neighbour. Each particle finds the optimal solution in search space. To enhance learning factors, minimise weights, and ANN architecture, the PSO algorithm has been used [17]. Consequently, it is usually used in engineering fields with practical applications. For example, PSO is commonly used for price forecasting, clustering, planning optimisation parameter optimisation, image processing and the medical field [18].

The PSO algorithm suffers from premature convergence and diversity problems. If PSO parameters are not properly set, then there are chances that it can get trapped in local optimum due to lack of local exploitation, global exploration and diversity issues in the search space [19]. To solve the combinatorial optimization problems, multiple modified PSO variants are proposed in [20,21], such as multi-objective optimization [22], constraint optimization [23], opposition-based variant [24], adaptive inertia weight [25] and mutation operator [26]. In addition, many modifications have been made to the PSO algorithm to improve its convergence. Inertia weight is introduced and gives more control over the particle capabilities for exploration (searching the entire space to gain a solution) and exploitation (searching the neighbourhood of the fittest solution).

Tizhoosh [27] introduced the concept of Opposition-Based Learning (OBL), which has so far been used to accelerate backpropagation learning on neural networks [28] and reinforcement learning. The purpose of OBL is to check the estimate (random guess) and its corresponding opposite estimate (opposite guess) at the same time to obtain a more approximate value for the current candidate solution [29]. It is used to generate opposite populations that inspire the real world’s opposite. The probability of finding the best solution is increased through the opposite solution. It improves algorithms’ performance and finds the best possible solution in search space [30]. OBL is used in many areas, such as agriculture for preserving water crops, soil purification, medical fields for disease diagnosis, scheduling of agricultural work [31]. It is applied in many well-known algorithms such as the Bat algorithm [32,33], PSO [34], Grey Wolf Optimizer (GWO) [35], Harmony Search (HS) [36], Differential Evolution (DE) [37] and Artificial Neural Networks (ANNs).

The distribution of random numbers is divided into three major categories: probability sequences, i.e., Exponential, Beta, Gamma, Lognormal, Quasi-random sequence, i.e., Halton, Van der corput, Hamersley, Sobol, Faure [38], and the pseudo-random sequences Multiply with-carry, Linear congruential generator, Mersenne twister, Philox, and Threefry [39]. A probability sequence is a sequence of continuous probabilities according to a particular distribution’s scale factor and shape [40]. Pseudo-random sequences and quasi-random sequences outperform for a globally optimal solution due to cover all search spaces. We used QRS (Sobol sequence (S), Halton sequence (H), Faure sequence (F), Gaussian (G), and Lognormal (LN)) for the initialisation of the population.

This behavior is more persevering and intolerable for multimodal problems, as it contains many local and global optimums. The most considerable factor for this deprived performance can be the insufficient distribution of the population in the search area, i.e., to conclude that if the initial population does not search the complete search space adeptly, it is difficult to locate the robust solution points, and thus the results omit the global best solution. This problem can be overcome by adding the most structured and organized random distribution for population initialization. Random number sequences vary with respect to the nature of their morphological design, i.e., quasi-random sequences, pseudo-random sequences, and probability distribution. Due to this fact, a novel initialisation population method used a Quasi-random sequence (Faure) to create the initialisation of the swarm, and through the opposition-based method, an opposite swarm was generated. We proposed an Opposition rank-based inertia weight approach to adjust inertia weights of particles to increase the performance of the standard PSO. The proposed algorithm (ORIW-PSO-F) has been tested to optimise the weight of the feed-forward neural network for fifteen data sets taken from UCI.

This paper proposes new variants of the PSO algorithm, Faure, with opposition-based PSO-ranked inertia weight (ORIW-PSO-F). The proposed algorithms improve global search ability and solve real-world classification problems. The proposed algorithms have two main effective improvements in initialisation strategies (Faure) with Opposition based learning and rank-based inertia weight. We present a new Quasi-random initialisation strategy (Faure) to initialise the search space particles, and opposite particles generated using Opposition-based learning. Particle inertia weight was updated with opposition rank-based inertia weight-balancing exploration and exploitation. We trained artificial neural networks using pseudo-random sequences on real-world classification problems. From the well-known repository UCI, fifteen data sets were taken in order to compare the performance of classifiers.

The simulation results show that the proposed variant ORIW-PSO-F provided better results as compared to ORIW-PSO, ORIW-PSO-S, ORIW-PSO-H, ORIW-PSO-LN and ORIW-PSO-G. The proposed study is useful in a wide range of computer domains, including neural network training, classification problems, data mining, image processing, min-max problems, game, single processing, multi-objective optimisation, and complex real-world optimisation. It is also applicable to solving most numerical optimisation problems and problems converted to optimisation problems.

The contributions of this work are summarized as follows:

• To propose a novel initialisation population method using a Quasi-random sequence (Faure) to create the initialisation of the swarm, and through the opposition-based method, an opposite swarm is generated and the proposed Opposition rank-based inertia weight approach adjusts the inertia weights of particles;
• To find the best accuracy and compare its result with the previous state-of-the art approaches.

The rest of the article is structured as follows: Section 2 discusses Materials and Methods, including an initialisation strategy, opposition rank inertia weight and the training, and basic PSO working. Section 3 provides the results and discussion. Section 4 describes conclusions and possible work, followed by the references section.

2. Materials and Methods

2.1. Related Work

In [39], the authors proposed a multi-mean PSO algorithm known as the MMPSO algorithm for training multi-layer feed-forward neural networks (MLFNN). MMPSO finds better solutions than PSO and MMPSO has multiple swarms to find the best solution that is better than PSO.

The authors [40] introduced a new, modified PSO algorithm with two main modifications. The first, known as a self-adaptive parameter, and the second, a strategy-based method known as the SPS-PSO algorithm, optimise feed-forward neural networks by feature selection. The SPS-PSO algorithm was applied to deal with the large-scale FFNN optimisation problem and reduce the computational complexity. The authors present a new variant of PSO (NMPSO) to solve nonlinear pattern classification problems [41]. The proposed methodology structure is an ANN that offers optimal precision for a particular problem. In addition, this study introduces a new method for selecting the maximum number of neurons (MNN). The architecture simultaneously develops transfer function types and synaptic weights. The proposed method was tested to accuracy by solving synthetic pattern recognition problems. Furthermore, the artificial neural network designed using the proposed method was compared with ANN designed manually using backpropagation and learning algorithms of Levenberg–Marquardt.

Experiments were performed using 10 datasets from UCI to check the performance of the MMPSO algorithm [42]. As a result of the experiment, it was shown that the proposed algorithm was executed more efficiently than other algorithms. An improved variant of the PSO algorithm called LPSONS was presented in [43] to increase the optimisation speed of the standard PSO algorithm to train ANNs. The proposed algorithm implemented the PSO velocity operator with the Mantegna–Lévy distribution to improve the diversity of the population and increase accuracy. Bottom of Form The proposed LPSONS algorithm was used to optimise the feed-forward multi-layer perceptron (MLP) ANN training.

In [44], the authors presented the partial opposition-based learning with PSO, or POPSO, algorithm to increase the performance of basic PSO. Partial opposition-based learning (POBL) generates the opposite swarm of the original swarm. POPSO is used to train the MLFNN for mining medical data classification problems. Compared to all other algorithms, POPSO provides a better compromise between sensitivity and specificity when classifying medical datasets. An ANN trained with the PSO algorithm has been used to distinguish dengue hemorrhagic fever (DHF) and dengue fever (DF) patients from patients recovering or not who have Parkinson’s disease [45]. Finally, NNPSO was tested with a multi-layer neural network feed-forward network (MLPFFN) classifier to classify dengue fever patients from recovered or non-recovered patients.

In [17], the authors presented the WELL sequence, also known as the Well Equidistributed Long-period Linear with Particle swarm optimisation (WELL-PSO) to overcome the limitation of the basic PSO algorithm. A novel quasi-random sequence initialisation scheme, the WELL sequence is used to generate the initial population. The proposed method also trains NN and offers better results than existing training algorithms (with basic PSO methods and improved variants). The experimental results show that WELL-PSO performs better on real-world classification problems than improved variant and standard method PSO. PSO algorithms were combined with the forward feedback of the neural network in the Cleveland Clinic database to reduce the 13 effective attributes to 8 factors and optimise the accuracy and cost [46]. The researchers used the four different research classification methods, and the results show that the feature selection in the neural network FFBP and PSO algorithm is more effective. A modified PSO algorithm combined with a new training algorithm was proposed by [47] for the time series problem. The training algorithm has no exploded or vanished gradient problem because it does not require gradients. They compared the accuracy of the proposed learning algorithm using a deep recurrent neural network with LSTM and PSGM ANNs on ten-time series. The forecasting performance of the proposed algorithm was superior to the other methods.

Khan et al. [48] proposed Advance Particle Swarm Optimization (APSO) using NN to reduce training time and improve classification accuracy. In advance, Particle Swarm Optimization (APSO) inertia weight is updated with constriction factor to avoid the local optimal problem. The accuracy of the proposed algorithm was also checked with the different numbers of a neuron [49]. The proposed algorithms perform faster convergence than backpropagation neural networks. For short term price forecasting (STPF) and classification purposes, the authors presented Fuzzy adaptive particle swarm optimisation (FA-PSO) with Feed Forward Neural Networks (FFNN) in [50]. To prevent local optima, the proposed algorithm uses dynamic inertia weight. Weights and biases were constructed using FAPSO for FFNNs with fixed architecture. To predict the price of power, the proposed method is used.

In [51], the authors present centripetal accelerated particle swarm optimisation (CAPSO) combined with advanced PSO and Newton motion laws to evolve accuracy and ANN learning. The authors trained a feed-forward multi-layer neural network (FFNN) with the CAPSO algorithm to solve classification problems for diagnosing nine medical diseases. The CAPSO algorithm exhibits superior classification accuracy compared to most of the well-known algorithms used to diagnose nine medical disorders. Wahab et al. [52] presented a method to train an ANN using the PSO algorithm to identify damage to structures. The proposed algorithm was used to overcome ANN limitations by reducing the computation time by applying the gradient descent method used in neural network training. Numerical and experimental models using various damage conditions were used to evaluate the effectiveness of the proposed algorithm. The proposed algorithm ANN-PSO easily found damaged locations. Furthermore, PSO and its various improved variants have been successfully applied in energy saving domains for multiple reasons, including the appropriate size adjustment for energy systems with the use of PSO.

2.2. Research Methodology

This paper proposes new variants of Faure with opposition-based PSO ranked inertia weight (ORIW-PSO-F) to balance exploration and exploitation and prevent stuck local optima. We proposed two modifications to PSO algorithms: initialisation strategy is Faure initialisation techniques with OBL, and opposition rank inertia weight is adjusting inertia weight of the particle. The initialisation strategy uses the QRS approach and OBL to initialise the initial population in PSO algorithms.

In this paper, there are three main contributions: initialisation strategies, Opposition-based learning, and Opposition Rank-based inertia weight. OBL is used to generate the opposite swarm of the current swarm and enhance the performance of algorithms. In OBL, we used jumping probability 0.3 for opposite population generation. It must be pointed out here that all variables were dynamically generated while accumulating the opposite population due to generational jumping. In the current set, each variable used maximum and minimum values to calculate the opposite points instead of the defined boundary spacing. Opposition Rank-based inertia weight adjusted for each particle according to its fitness rank. The lowest fittest particle had maximum inertia weight that moved fast compared to the fittest particle.

2.3. Random Number Generator

A set of numbers that occur in an order in which values cannot be predicted based on the past and present and the value cannot be predicted based on a specific uniformly distributed set of numbers is called a random number. Random numbers are generated in a uniform distribution by the built-in library function Rand(x_min, x_max). The continuous uniform probability density function determines the effect of uniformity for all sequences [53]. It generates a sequence based on the probability density function. The probability density function is defined as:

$$f\left(t\right)=\{\begin{array}{c}\frac{1}{p-q} for p<t<q\\ 0 for t<p or t>q\end{array}$$

(1)

where p and q are the maximum likelihood parameters. The f(t) value is useless at the boundary between p and q, as it has no effect on the integral of f(t) dt in any range. The score likelihood function calculates the maximum likelihood parameter estimate. It is given as follows:

$$l(p,q\left|t)=n\mathit{log}(q-p)\right.$$

(2)

Flowchart chart show below in Figure 1.

2.4. Quasi-Random Sequence

Low-discrepancy techniques are also deterministic point generators that mean the point set with a high level of uniformity. The quasi-random sequence (QRS) approach was used to generate a low discrepancy set, and it was neither random nor pseudo-random. Quasi-random sequence generators reduce the discrepancy (non-uniformity) from the distribution with an equal component of points in each sub-cube of a uniform partition of the hypercube and fill the “holes” in any initial segment of the generated QRS. This technique avoids clustering and can accelerate convergence; however, quasi-random numbers are normally too uniform to pass randomness tests. QRS was used as an initial value for the global optimisation problem. QRS explores more space than a random sequence used in applied mathematics. QRS is used to initialise the population in optimisation algorithms. Famous quasi-random sequences such as Sobol, Halton, Faure, Gaussian, and Lognormal are used for the initialisation of population.

2.4.1. Sobol

An example of quasi-random sequences is the Sobol sequence. A set of direction numbers has to be required to generate a Sobol sequence. Sobol provides the liberty while selecting the initial direction numbers. By using Sobol sequences, we can generate results for selected dimensions. Base 2 generates a finer uniform partition of unit intervals for these sequences. Sobol sequence is generated through the following equation and Figure 2:

$$c=c_1{2}^0+c_2{2}^1+c_3{2}^2+\dots +c_n{2}^{n-1}$$

(3)

2.4.2. Halton

Each dimension of the Halton sequence uses coprime bases, also known as van der Corupt sequences. Halton sequences are of low discrepancy and deterministic, used in the Monte Carlo simulation. It is an improved variant of the van der Corput sequence as shown below in Figure 3.

2.4.3. Gaussian

Another name of the normal distribution is the Gaussian distribution; mostly, data occur near the mean, and hence it has a bell curve. For the representation of real-valued random variables we used continuous-distribution-type Gaussian distribution, as shown in Figure 4.

2.4.4. Lognormal

The logarithm of the log normal distribution follows a normal distribution and type of probability distribution. The log normal distribution is skewed to the right and applicable when the growth rate is positive because the logarithm of value exists only when positive. As shown in Figure 5.

2.4.5. Faure

The Faure sequence is an approach to generating a Low Discrimination Sequence; it enhances the most basic idea of the van der Corput sequence for higher dimension. The basic method is to generate the sequence of the van der Corput methodology. Distribution of 100 particles in the search space [0, 1] is shown in Figure 6 for Quasi sequence uniform distribution while pseudo sequence is shown in Figure 7.

2.5. Opposition Based Learning

The optimisation process will end when the optimal solution is near the random guess and fast convergence. The optimisation process will take a lot of time when the optimal solution is far from random guesses, and convergence will be slower. Random guesswork and its counter-guessing can be generated simultaneously to increase the availability of the optimal solution [54], and whereas a metaheuristic algorithm enhances a problem with the best solution, the initial solution is usually randomly generated [55]. However, using OBL achieves a better candidate start date and has a better chance of finding a better area, even without prior knowledge. The main idea behind OBL is to make the current estimate x′ that is randomly generated for each solution x of a given problem. To find the opposite value x′ of current value x, we calculate it as follows:

$$x^{\prime }=a+b-x$$

(4)

where [a,b] are the intervals of real number x.

Let us assume x_i = (x₁, x₂, x₃, …, x_d) are points in search space d dimension with interval x_i belonging to {a_i, b_i}, so the opposite point is as follows:

$$x_i^{\prime }=a_i+b_i-x_i$$

(5)

Let us assume, given g(x) is the evolution function and the unknown function is f(x); if f(x) is better than f(x′), then learning continues with x; otherwise, we continue with x′.

The OBL method used in the proposed algorithm differs from existing OBL-based algorithms. The existing OBL method first randomly initialises the population and then calculates the opposite population. The first population is initialised through Quasi-random sequence (Faure) in proposed algorithms and calculates its opposite.

2.6. Opposition Rank Base Inertia Weight

All the inertia weight has a significant influence on the performance of the standard PSO algorithm [56]. Inertia weight is the effective parameter that maintains the velocity of particles. Inertia weight is vital for balancing local search, known as exploitation (for lower values), and global search, also known as exploration (for higher values). Researchers have proposed many variants of it. This paper adopted a rank-based strategy to solve the problems. Particle inertia weight was updated with the rank-based inertia weight strategy. Opposition Rank-based strategy adjusted inertia weight according to particle fitness value and assigned a fitness rank to each particle. The most suitable particles near the best position will move slowly, while the fast-moving particles far away from the best position will continue to explore. The suitable particle is selected from either current or their opposite. It has improved local and global search at the same time. To find the rank base inertia weight of each particle, we calculated it as follows:

$$W_{\left(i\right)}\left(t\right)=W_{max}+\left(\left(\left(W_{min}-W_{max}\right)\right)/n\right)\ast R_{\left(i\right)}\left(t\right)$$

(6)

where n is the size of the population and R_(i)(t) is the fitness rank of i particle. The slowly moving particles are the fittest particles that have the lowest inertia weight and fitness rank 1. The fast-moving particle has the highest fitness value rank, and the highest inertia weight is assigned. Maximum inertia weight (W_max) was set at 0.9, and 0.4 was the minimum inertia weight. Below Figure 8 show the feed forward neural network.

3. Results and Discussion

The proposed approach of ORIW-PSO-F was implemented in Matlab 2016 and a computer with 2.00 GHz along with the 8 GB RAM, Core™ i3-5005U CPU processor specification. We conducted experiments using fifteen datasets that have been taken from the UCI repository.

We separated the datasets into two parts: training and testing. The size of the training portion is 70% of the dataset and the testing portion is 30% of the dataset. On the range [50, 50], the initialization of the training weights is random. The dataset’s features are shown in

Table 1. Datasets detail.

Sr. No	Dataset	No of Attributes	Number of Labels	Number of Records
1	Iris	4	3	150
2	Wheat seed	7	3	210
3	Pima India Diabetes	8	2	768
4	Heart Disease	13	2	270
5	Wisconsin Breast Cancer	10	2	699
6	Vertebral	6	2	310
7	Wine	13	3	178
8	Haberman’s survival	3	2	306
9	Balance scale	4	3	625
10	Blood Transfusion	4	2	748
11	Sonar	60	2	208
12	Bank Note Authentication	4	2	1372
13	Ionosphere	34	2	351
14	Liver Disorder	6	2	345
15	Car Evaluation	6	4	1728

. These characteristics include the total number of instances that participated in each dataset, the total number of features, and the number of classes in each data set (such as a binary class problem or multiclass problem).

The proposition of this study continues to observe whereby the unique characteristics of experimental results rely on dimensions of these standard data sets. In the experiments, three simulation experiments were performed, where the following features of ORIW-PSO-F were observed: the effect of using different Initializing PSO approaches, the effect of using different Dimensions for problems, and a comparative analysis.

Feed-forward neural network weights are trained on a particle swarm optimisation algorithm (PSO), Sobol with PSO (PSO-S), Halton with PSO (PSO-H), Log normal with PSO (PSO-LN), Gaussian with PSO (PSO-G), and Faure with PSO (PSO-F). PSO-F performs well on these datasets and shows good results. To evaluate the performance, the PSO-F algorithm was compared with a variant of PSO such as standard PSO, PSO-S, PSO-H, PSO-G, and PSO-LN on fifteen real data sets extracted from UCI. Simulation results show that neural network training using the PSO-F algorithm performed well and provided better accuracy than other PSO approaches. After simulation, the result was excellent in training the NN using the PSO-F algorithm, and shows better precision and accuracy than traditional approaches. The accuracy results of classification problems are depicted in Figure 9, and an accuracy graph represents the same figure for fifteen data sets.

A one-way ANOVA test with a significance level of 0.05 was implemented on testing the accuracy of six PSONN approaches. The result of the one-way ANOVA test is depicted in

Table 2. One-way ANOVA result test of QRS with PSO approaches.

Parameter	Relation	Sum of Squares	df	Mean Square	F	Significance
Testing Accuracy	Between groups	903.2158	5	180.6432	2.334319	0.049042

. In Table 2, the value of significance is 0.04902, which is less than 0.05, indicating a significant difference between all PSONN variants with a 95% confidence level. Therefore, PSONN variants were significantly different from each other. The one-way ANOVA test results are represented in Figure 10, which shows PSO-F has significantly better results than other PSO approaches. The graph of the ANOVA test shows the PSO-F is significantly distinct from all other PSO approaches.

The weights of the feed-forward neural network were trained on opposition-based PSO (OPSO), Sobol with opposition-based PSO (OPSO-S), Halton with opposition-based PSO (OPSO-H), log normal with opposition-based PSO (OPSO-LN), Gaussian with opposition-based PSO (OPSO-G), and Faure with opposition-based PSO (OPSO-F). We prepared a feed-forward neural network using the weight optimisation process. The performance of OPSO, OPSO-S, OPSO-H, OPSO-G, OPSO-LN OPSO-F and state-of-the-art NN algorithms were tested on 15 well-known datasets. To evaluate the performance of the OPSO-F algorithm, it was compared with variants of PSO such as standard OPSO, OPSO-S, OPSO-H, OPSO-G, and OPSO-LN on fifteen real data sets extracted from UCI. The detail of the fifteen data sets are present in Table 1. OPSO-F is well-performing on these datasets and shows good result. Simulation results in Figure 11 show that neural network training using the OPSO-F algorithm performed well and provided better accuracy than other PSO approaches. Testing accuracy graph of fifteen datasets are represented in Figure 11.

A one-way ANOVA test with a significance level of 0.05 was implemented on the testing accuracy of six PSONN approaches.

Table 3. One-way ANOVA result test of QRS with Opposition-Based PSO approaches.

Parameter	Relation	Sum of Squares	df	Mean Square	F	Significance
Testing Accuracy	Between groups	818.691	5	163.738	2.334012	0.0494

depicts the results of the one-way ANOVA test. The significance value in Table 3 is 0.0494, which is smaller than 0.05, showing that there is a significant difference between all PSONN variants with a 95% confidence level. Therefore, PSONN variants are significantly different from each other. Figure 12 depicts the one-way ANOVA test results, which shows OPSO-F significantly has better results than other PSO approaches. The graph of ANOVA test show the OPSO-F is significantly distinct from all other PSO approaches.

The weights of the feed-forward neural network were trained using opposition-based PSO ranked inertia weight (ORIW-PSO), Sobol with opposition-based PSO ranked inertia weight (ORIW-PSO-S), Halton with opposition-based PSO ranked inertia weight (ORIW-PSO-H), log normal with opposition-based PSO ranked inertia weight (ORIW-PSO-LN), Gaussian with opposition-based PSO ranked inertia weight (ORIW-PSO-G), and Faure with opposition-based PSO ranked inertia weight (ORIW-PSO-F). We prepared a feed-forward neural network using the weight optimisation process. The performance of ORIW-PSO, ORIW-PSO-S, ORIW-PSO-H, ORIW-PSO-G, ORIW-PSO-LN, ORIW-PSO-F and state-of-the-art NN algorithms have been tested on 15 well-known datasets. To evaluate the performance of the proposed ORIW-PSO-F technique, it was compared with variants of PSO such as ORIW-PSO, ORIW-PSO-S, ORIW-PSO-H, ORIW-PSO-G, and ORIW-PSO-LN on fifteen real data sets extracted from UCI. The detail of these fifteen data sets is presented in Figure 13, where ORIW-PSO-F is well-performing on these datasets and shows good results. Simulation results in Figure 13 show that neural network training using the ORIW-PSO-F algorithm performed well and provided better accuracy than other PSO approaches. After simulation, the result is excellent in training the NN using the ORIW-PSO-F algorithm, and shows better precision and accuracy than traditional approaches. The ORIW-PSO-F algorithm can be effectively used for real-world complex statistical problems and data classification problems in the future. The accuracy results of classification problems are depicted in Figure 14, and the accuracy graph is represented in the same figure for fifteen data sets.

The one-way ANOVA test with a significance level of 0.05 was implemented on testing accuracy of six PSONN approaches.

Table 4. One-way ANOVA result test of QRS with opposition-based PSO rank based inertia weight approaches.

Parameter	Relation	Sum of Squares	df	Mean Square	F	Significance
Testing Accuracy	Between groups	818.095	5	163.619	2.334622	0.04804

shows the results of the one-way ANOVA test. The significance level in Table 4 is 0.04902, which is less than 0.05, indicating that there is a significant difference with a 95% confidence level in all PSONN variants. Therefore, PSONN variants are significantly different from each other. Figure 10 depicts the one-way ANOVA test results, which shows ORIW-PSO-F has significantly better results than other PSO approaches. The graph of the ANOVA test shows that the ORIW-PSO-F is significantly distinct from all other PSO approaches.

4. Conclusions

PSO has been widely used in various fields to solve real nonlinear complex optimisation problems. It still requires extensive testing to improve its performance, and researchers have proposed several variants of PSO. This paper gives exhaustive detail for the training of feed-forward neural network (FFNN) utilised for different PSO approaches with QRS (Faure) and opposition rank-based inertia to solve premature convergence and local optima problems for best results. In the proposed technique, the initialisation scheme of QRS was used with opposition base method to generate the initial population. Opposition rank-based inertia is used to balance exploitation and exploration searchability. The accuracy results show that the proposed technique ORIW-PSO-F is better than other improved variants. The results illustrate how the proposed techniques affect convergence speed and diversity. Although the primary purpose of this research is to develop a future direction of our work, it could be applied to other stochastic-based meta-heuristic algorithms using mutation operators with this initialisation method. Similarly, this approach and its future variants can also be observed in the energy-saving application area.