Intelligent Multi-Strategy Hybrid Fuzzy K-Nearest Neighbor Using Improved Hybrid Sine Cosine Algorithm

Abstract

The sine and cosine algorithm is a new simple and effective population optimization method proposed in recent years that has been studied in many works of literature. Based on the basic principle of the sine and cosine algorithm, this paper fully studies the main parameters affecting the performance of the sine and cosine algorithm, integrates the reverse learning algorithm, adds an elite opposition solution and forms the hybrid sine and cosine algorithm (hybrid SCA). Combined with the fuzzy k-nearest neighbor method and the hybrid SCA, this paper numerically simulates two-class datasets and multi-class datasets, obtains a large number of numerical results and analyzes the results. The hybrid SCA FKNN proposed in this paper has achieved good accuracy in classification and prediction results under 10 different types of data sets. Compared with SCA FKNN, LSCA FKNN, BA FKNN, PSO FKNN and SSA FKNN, the prediction accuracy is significantly improved. In the Wilcoxon signed rank test with SCA FKNN and LSCA FKNN, the zero hypothesis (significance level 0.05) is rejected and the two classifiers have a significantly different accuracy.

1. Introduction

The swarm intelligence algorithm (SI) has gained attention from many researchers in various field of sciences. SI is currently being used to provide solutions to various optimization problems. Several applications of swarm intelligence include material technology [1,2,3], biological system modeling [3], train assembly, high-performance graphics card [4], path planning [5] and robot control [6,7]. SI is based on the collective behavior of the elements that self-organize in order to get exposed with the solution of the optimization problem. Examples of popular SI algorithms include particle swarm optimization (PSO) [8], artificial bee colony (ABC) [9], gravitational search algorithm (GSA) [10] and whale optimization algorithm (WOA) [11]. One of the main challenges in SI is the lack of profound theoretical analysis, which requires a solid mathematical foundation that includes a proper analysis that assesses the robustness, computational complexity and parameter setting. All of these analyses are required to ensure that SI can avoid converging to a local minimum solution. Note that the local minimum solution will affect the optimality of the solution to the respective optimization problem.

The sine cosine algorithm (SCA) [12] is a new SI algorithm proposed by Mirjalili in 2016. SCA was inspired by a mathematical model of the sine cosine function used to make the oscillation of the solution converge towards the optimal solution. The random and adaptive parameters in the algorithm have the ability to balance both exploration and exploitation during solution searching. Several advantages of SCA include very few parameters, easy implementation, a simple structure, a fast convergence speed, strong parallelism and universality and a better performance in practical applications. Therefore, it has attracted extensive attention from scholars in recent years.

In [13], a SCA with nonlinear decreasing conversion parameters was proposed. The change in parameter r1 is controlled by a parabolic function and exponential function, respectively. The experimental results show that the adjustment of parameter r1 by the exponential function can better balance the global and local exploration of the solution. The study in [14] proposed a method of combining quantum computing with SCA by using quantum bits, rotating gates and a non-gate, where each gate has their own specialization in exploring the solution. The proposed SCA was reported to be very effective and accurate. The study in [15] introduces the method of reverse learning to generate reverse solutions for the current individual, which expands the exploration of solution space. The study in [16] proposed a hybrid gray wolf with SCA that uses the sine cosine update formula to improve the moving direction and speed of the head wolf. By combining the benefit of SCA, the hybrid algorithm improves drastically in terms of exploration and exploitation. Another interesting work by [17] was carried out, where SCA was combined with a differential evolution algorithm (DEA). In this context, SCA was reported to help DEA jump out from the local optimal solution region. However, the application of the SCA in the above literature failed to classify a more general dataset. The test of the robust metaheuristics depends on the ability of the algorithm to classify non-bias datasets.

In 2017, Rizk M and Rizk-Allah [18] proposed a sine cosine algorithm (MOSCA) based on a multi-orthogonal search strategy to solve engineering design problems. The proposed MOSCA improves the defects of unbalanced exploration and premature convergence of the conventional SCA. MOSCA utilizes SCA during the exploration phase and uses a multi-orthogonal search strategy to find the optimal solution in the search space. MOSCA was reported to obtain a better speed of convergence with a higher solution accuracy. In another development, Elaziz et al. [15] proposed SCA based on reverse learning (OBSCA). A reverse learning strategy is an important method used to enhance the performance of the stochastic optimization algorithm. By selecting the value of the objective function according to greedy selection at the current solution and reverse solution, OBSCA enhances the diversity of the population and improves the ability of the algorithm to approach the global optimal solution. This experiment highlights the robustness of OBSCA in terms of convergence.

In 2017, Songjin and Wen [19] proposed an improved SCA (ISCA) for solving high-dimensional optimization problems. Inspired by PSO, the ISCA algorithm introduces inertia weight to improve the convergence accuracy and increase the convergence speed of the SCA. At the same time, it adopts a reverse learning strategy to generate initial individuals to improve the diversity and reconciliation quality of the population. The experimental results showed that, compared with the basic SCA, ISCA has a better optimization performance in a high-dimensional test function. In 2018, Nenavath and Jatoth [20] proposed a hybrid SCA-DE algorithm based on differential evolution to solve optimization problems and target tracking problems. The experimental results show that the hybrid SCA-DE algorithm has a higher convergence accuracy and faster convergence speed than basic SCA. In 2021, Wu et al. [21] proposed a LSCA method and FKNN method to solve biomedical problems. Compared with other methods, the proposed LSCA obtained acceptable results but the accuracy of this method still requires improvement.

In this paper, we capitalize on the mathematical properties of the SCA to balance the global and local exploration of the algorithm during the searching process. This can be achieved by adaptively changing the amplitude of the sine function and cosine function until the SCA converges towards the global optimal solution. In addition, reverse learning will be used to provide a jump mechanism to the SCA so that it can avoid a potential unwanted local solution. Both methods will be integrated into a fuzzy k-nearest neighbor (FKNN) that has the capability to classify real life datasets. Thus, the contributions of this paper are as follows:

• In this paper, reverse learning will be implemented into an SCA model to form a hybrid SCA. In this context, the adaptive weight coupled with the reverse learning alter the position of the solution towards the global solution.
• The proposed hybrid SCA will be implemented into a fuzzy k-nearest neighbor (SCA-FKNN). In this context, the proposed SCA-FKNN has the ability to avoid local convergence by jumping out of the current non-optimal solution.
• The performance of the proposed SCA-FKNN will be tested using various real life datasets. SCA-FKNN will be evaluated according to the various performance metrics, such as accuracy, precision, sensitivity, specificity, Mathews correlation coefficient and Wilcoxon signed rank test. In addition, the proposed SCA-FKNN will be compared with the existing conventional state-of-the-art classifier.

The rest of this paper is organized as follows. Section 2 introduces in detail the content of the SCA model and FKNN classifier. Section 3 presents in detail the process of forming the hybrid SCA FKNN model based on the SCA model and FKNN classifier by adding a parameter adjustment and reverse learning mechanism. Section 4 introduces and analyzes 10 different data sets and evaluation indicators. Section 5 shows the prediction and classification results of 10 types of data sets under five models with extensive comparison analysis. Section 6 describes conclusions and further research.

2. Background

2.1. Sine Cosine Algorithm

The sine cosine algorithm is an algorithm based on the mathematical characteristics of sin and cos. It updates individuals through the changes in sine and cosine functions. In SCA, it is assumed that, in j-dimensional space, the population size is n, and that, in each iteration, the location update mode of the i-th individual is

$${\mathrm{X}}_i^j(t+1)=\left\{\begin{array}{c}{\mathrm{X}}_i^j\left(t\right)+r_1\times sin\left(r_2\right)\times \left|r_3{X}_{best}^j-{\mathrm{X}}_i^j\left(t\right)\right|r_4<0.5\\ {\mathrm{X}}_i^j\left(t\right)+r_1\times cos\left(r_2\right)\times \left|r_3{X}_{best}^j-{\mathrm{X}}_i^j\left(t\right)\right|r_4\ge 0.5\end{array}\right.$$

(1)

where $X_i^j$ is the position of the i-th individual in the j dimension of the t iteration; $X_{best,j}$ is the optimal position in the j dimension of the position $X_i$ of the i-th individual; $r_2$, $r_3$ and $r_4$ are random numbers subject to uniform distribution, $r_2\in [0,2\pi ]$, $r_3\in [0,2]$ and $r_4\in [0,1]$; $r_1$ is the control parameter.

$$r_1=a\ast (1-\frac{t}{MaxFEs})$$

(2)

where $MaxFEs$ represents the maximum number of iterations and a is a constant number and is equal to 2.

The fluctuation amplitude of $r_1\times sin\left(r_2\right)$ and $r_1\times cos\left(r_2\right)$ (sine and cosine parameters) gradually attenuates with the increase in iteration times. Its values are in the range of $(1,2]$ and $[-2,-1)$. The algorithm performs a global search in the solution space, and the algorithm performs a local development in the range of $[-1,1]$. The SCA algorithm flow is shown in Figure 1.

2.2. Fuzzy K-Nearest Neighbors (FKNN)

As one of the simplest classifiers, KNN mainly infers the class of the sample according to the classes of the K training samples closest to the sample to be classified. The default of this method is that each of the k samples have the same weight, which is not the case. The KNN algorithm (nearest neighbor method) was first proposed by Covcr and Hart in 1967. Many researchers have conducted in-depth theoretical research and development due to the low error rate of the nearest neighbor method, which makes it one of the important methods of pattern classification.

The FKNN algorithm (fuzzy k-nearest neighbor algorithm) was proposed by Keller et al. in 1985 [22]. He assigned different weighting coefficients to k-nearest neighbors, and then used the fuzzy decision-making method to calculate the class label with the largest coefficient as the category of test data. Because the weight coefficient based on distance is used, the recognition effect is improved. Nevertheless, the selection of fuzzy k-nearest neighbor parameter K has a great impact on the recognition effect. Choosing appropriate parameters plays an important role in improving the accuracy of the classification.

A fuzzy KNN algorithm is proposed based on the KNN algorithm. This method has the advantages of a high calculation accuracy and no data input assumption. It is a relatively mature classifier. For data sets, the membership of each member data to each class is calculated by Equation (3).

$$U_{i,k}=\left\{\begin{array}{c}0.51+\left(\frac{n_k}{K}\right)·0.49,k=Y_k\\ \left(\frac{n_k}{K}\right)·0.49,k\ne Y_k\end{array}\right.$$

(3)

where $i=1,2,3\dots N$ represents the i-th training sample and N represents the number of all training samples. $k=1,2,3\dots M$, where k represents the k-th class, and M denotes the number of classes. $U_{i,k}$ represents the member level of the i-th sample to the k-th class. K represents the present number of nearest neighbors, $Y_k$ represents the class of the i-th training sample and $n_k$ represents the number of the i-th training sample’s neighbors belonging to the k-class among the nearest K neighbors. Note that membership should meet the following:

$$U_k\left(x\right)=\frac{{\sum }_{j-1}^K{U}_{I_j,k}{\left(x-x_{I_j}\right)}^{\frac{2}{m-1}}}{{\sum }_{j-1}^K{\left(x-x_{I_j}\right)}^{\frac{2}{m-1}}}$$

(4)

where x stands for the test sample, $U_k\left(x\right)$ represents the test sample weight to the k-class, $j=1,2,\dots K$ represents the test sample’s j-th nearest neighbor, $I_j$ represents the i index corresponding to the j-th nearest neighbor in the training samples, $U_{I_j,k}$ is the membership degree, which is calculated by Equation (3), and $x-x_{I_j}$ represents the distance measurement. m stands for fuzzy strength, which is used to control the weight of each neighbor in the membership calculation, and its range is $[1,\infty ]$.

$$C\left(x\right)=\underset{k}{\mathrm{a}\mathrm{r}\mathrm{g} \mathrm{m}\mathrm{a}\mathrm{x}}{U}_k\left(x\right)$$

(5)

The calculation steps of FKNN are as follows in Figure 2.

The above FKNN solved the problem of multivariate classification and distance weight, and the SCA will deal with the problem of a low search efficiency of the optimal solution after the distance weight.

3. The Proposed Method

At the end of the iteration, SCA will conduct a small neighborhood search near the current global optimal location and constantly try to update the optimal solution. If the search process is far from the theoretical optimal solution, it is difficult for the algorithm to converge to the global optimal solution in a short time. Therefore, the current research papers are roughly divided into two ways to improve the convergence speed and accuracy of SCA. One way is to improve the convergence speed of SCA by changing Equation (1). The other is to improve the accuracy of SCA by adding reverse learning.

In the parameter adjustment mechanism, reference [23] has introduced the adaptive weight coefficient parameter adjustment mechanism, and this mechanism has achieved good results in solving the problem of jumping out of local convergence. Based on the parameter adjustment mechanism and combined with reverse learning, this paper forms an improved version of the sine and cosine algorithm with multiple strategies.

The combination of the swarm intelligence algorithm, lion swarm algorithm and reverse learning strategy further expands the search scope of the group, thus improving the problems of a slow convergence speed and insufficient accuracy of the group.

3.1. The Weight Factor

In this part, an adaptive weight w is used, which makes the individual position have a great impact on the individual moving direction and distance in the algorithm and effectively improves the ability of algorithm development. The value of $w^{t+1}$ in the latter iteration is 100 times that of the previous iteration $w^t$, with an obvious step search. The mathematical model [23] of w is

$$w=\mu \times sinh{(1-20\frac{t}{MaxFEs})}^8$$

(6)

where $\mu$ is the weight factor; in most cases, the value of $\mu$ is 0.5. Adding the weight parameter w to the sine cosine algorithm in Equation (1), we obtain:

$${\mathrm{X}}_i^j(t+1)=\left\{\begin{array}{c}{\mathit{w}(\mathrm{X}}_i^j\left(t\right)+r_1\times sin\left(r_2\right)\times \left|r_3{X}_{best}^j-{\mathrm{X}}_i^j\left(t\right)\right|)r_4<0.5\\ {\mathit{w}(\mathrm{X}}_i^j\left(t\right)+r_1\times cos\left(r_2\right)\times \left|r_3{X}_{best}^j-{\mathrm{X}}_i^j\left(t\right)\right|)r_4\ge 0.5\end{array}\right.$$

(7)

3.2. Reverse Learning

In SCA, the individuals of the population only rely on the current optimal solution to update their own state, so the algorithm is likely to fall into the local optimal state, resulting in the algorithm being unable to find a satisfactory solution. At this time, it is necessary to carry out a local mutation operation on the individual, and the individual reflects on the previous learning situation with the current learning results so as to increase the probability of escaping from the local area. The formula of reflective learning is

$$X_i^{\ast }=X_i^s+\omega \otimes (X_i^s-X_i^t)$$

(8)

where $X_i^s$ represents the position of individual i in the t-th iteration; $X_i^{\ast }$ represents the position after executing Equation (7); $X_i^{\ast }$ represents the new position generated through the reflection process; $\omega$ represents a learning factor, $\omega \in [-1,1]$; ⊗ indicates dot multiplication.

In order to prevent too much randomness in the process of reflection, the learning factor was compared with $\omega$. At the same time, in order to avoid the degradation of the learning ability and enhance the convergence of the algorithm, greedy learning was used to select the best algorithm according to the learning status before and after reflection.

$$\omega =C^{\left(\raisebox{1ex}{$-t$}\!\left/ \!\raisebox{-1ex}{$MaxFEs$}\right.\right)}\times \mathrm{c}\mathrm{o}\mathrm{s}\left(r_5\right)$$

(9)

where $r_5$ is a random number on $\left[0,\pi \right]$; C is a constant, and the effect is better when $C=100$.

In order to reduce the possibility of the algorithm deviating from the global optimal position, the evaluation of excellent algorithms is strengthened. It is very necessary to search the space around the volume, and this improvement can improve the efficiency of the algorithm and the ability to explore new solutions. This paper integrates the strategy of elite reverse learning into SCA. The information of the elite population was used to search the space of elite individuals and their reverse solutions.

The specific operations [24] were as follows:

• The individuals in the population were arranged after the implementation of formula $fitness$, where 10% of the excellent individuals were selected to form the elite population ${\mathbb{X}}_{best}$;
• Individual $X_{best}^i\in {\mathbb{X}}_{best}$ boundary $[l{b}_j^i,u{b}_j^i]$ and the dynamic boundary $[min\left(l{b}_j^i\right),$ $max\left(u{b}_j^i\right)]$ were calculated;
• The dynamic elite reverse population ${\mathbb{X}}_{best}^{\prime }$ of individual $X_{best}^i$ was generated according to Equation (10);
• If the reverse population ${\mathbb{X}}_{best}^{\prime }$ exceeded the limit of dynamic boundary $[min\left(l{b}_j^i\right),$ $max\left(u{b}_j^i\right)]$, it was replaced by a new individual randomly generated in the boundary;
• The top 50% from $[{\mathbb{X}}_{best},{\mathbb{X}}_{best}^{\prime }]$ was selected for the next generation according to $fitness$;
• Steps 2 and 5 were cycled until the stop condition was reached, and the algorithm ended.

The elite inverse solution was set in d-dimensional space. $X_{best}^{\prime }=\left(x_1^{\prime },x_2^{\prime },\cdots ,x_D^{\prime }\right)$ is the inverse solution of the elite individual. $X_{best}=\left(x_1,x_2,\cdots ,x_D\right)$ is the inverse solution of the current population. The inverse solution is defined as

$$x_i^{\prime }=k(l{b}_i+u{b}_i)-x_i$$

(10)

where $k\in \left[0,1\right]$ is a random number subject to uniform distribution. Multiple inverse solutions of the elite individual can be generated by using this coefficient.

The generated elite inverse solution increases the useful information of the population converging to the global optimum, strengthens the exploration of the neighborhoods around the optimal individual and improves the local development ability of the algorithm.

3.3. The Proposed Hybrid SCA FKNN Model

In this paper, the $Fitnes{s}_i$ is equal to ACC. ACC represents the accuracy of FKNN classification, which is obtained by k-cross-validation. In this paper, five-fold validation was used. After combining hybrid SCA and FKNN, the pseudocode of the hybrid SCA FKNN is shown in Algorithm 1. Figure 3 and Figure 4 show the operation flow of the whole hybrid SCA FKNN method in detail.

Algorithm 1: The hybrid SCA-FKNN.

while$t<MaxFEs$do

update $r_1,r_2,r_3,r_4$

$w_t=\mu \ast sinh{(1-20\frac{t}{MaxFEs})}^8$

if $r_4<0.5$ then

$X_i^s={{\mathrm{w}}_{\mathrm{t}}(\mathrm{X}}_i\left(t\right)+r_1\times sin\left(r_2\right)\times \left|r_3{X}_{best}-{\mathrm{X}}_i\left(t\right)\right|)$

$X_i^{\ast }=X_i^s+\omega \otimes (X_i^s-X_i^t)$

if $f\left(X_i^s\right)>BF$ then

if $f\left(X_i^s\right)<f\left(X_i^{\ast }\right)$ then

${\mathrm{X}}_i^{t+1}=X_i^{\ast }$

$X_{best}={\mathrm{X}}_i^{t+1}$

$BF=f\left(X_i^{\ast }\right)$

else

${\mathrm{X}}_i^{t+1}=X_i^s$

$X_{best}={\mathrm{X}}_i^{t+1}$

$BF=f\left(X_i^{\ast }\right)$

end if

end if

for i = 1 to $SupN$ do

generate random k, $X_i^{\prime }(t+1)=k(lb+ub)-X_i(t+1)$

end for

put all $X_i^{\prime }$ into train dataset as elite opposition solutions

else

$X_i^s={{\mathrm{w}}_{\mathrm{t}}(\mathrm{X}}_i\left(t\right)+r_1\times cos\left(r_2\right)\times \left|r_3{X}_{best}-{\mathrm{X}}_i\left(t\right)\right|)$

As the up, the same progress

end if

end while

4. Experiment and Discussion

4.1. Experiment Setup

In this section, the components of the experiment will be described in detail. The purpose of this experiment is to illustrate that the hybrid SCA FKNN method proposed in this paper can be used in two or more types of data sets and can achieve good numerical results, with strong adaptability and accuracy. In order to ensure the reproducibility of the experiment, the experimental setup will be shown below.

4.2. Benchmark Datasets

The datasets used in this paper were all open source datasets; for details of the datasets used in this paper, please visit this website: https://archive.ics.uci.edu/ml/index.php (accessed on 30 April 2022). For all datasets, the feature was normalized to $[-1,1]$ using maximum and minimum normalization. In order to better show the numerical results of the method proposed in this paper, this paper focused on the numerical experiments on two-class data sets and multi-class data sets.

In order to reflect the wide adaptability of the methods proposed in this paper, this paper selected 10 kinds of data sets from different application scenarios and different data types. The 10 kinds of data sets involve a variety of use scenarios with practical life significance, such as medical treatment, daily necessities, automobiles, etc. From

Table 1. The two-class-dataset-related information.

	Categories	Samples	Features	Positive	Negative
Bupa	2	345	6	145	200
Hepatitis	2	155	19	32	123
SPECT	2	267	22	212	55

and

Table 2. The multi-class-dataset-related information.

Datasets	Categories	Samples	Feartures	Positive	Negative
Caesarian section classification dataset	2	80	4	34	46
Indian liver patient dataset (ILPD)	2	583	10	415	167
Glass identification dataset	7	214	9	69 (class 1)	145 (other classes except positive)
User knowledge modeling dataset	4	403	5	102 (class 1)	301 (other classes except positive)
Breast tissue dataset	6	106	9	20 (class 1)	86 (other classes except positive)
Car dataset	4	1728	6	1209 (class 1)	519 (other classes except positive)
QCM sensor alcohol dataset	5	125	15	24 (class 3)	101 (other classes except positive)

, it can be seen that the sample size, characteristics and categories of the 10 types of data sets cover different levels. This paper verifies the effectiveness of the method proposed in this paper from various angles according to different conditions and different use scenarios of the data sets.

For two-class data sets, this paper considered the following three data sets. The basic situation of these three data sets is shown in Table 1. Table 1 describes the indicators, such as the number of data label categories, data scale and data feature quantity, of the following three datasets.

For multi-classes datasets, in order to further verify the effectiveness of the method proposed in this paper, this paper retrieved the following eight types of data sets on the open data platform of the University of California for numerical experiments. The contents of the relevant data sets are described in Table 2. These data sets involve multiple areas of life, which is more convincing for verifying the effectiveness of the method. The relevant data categories range from the least to the most, and the diversity of data features ranges from small sample data to large sample data.

Table 2 describes the basic information of seven multi-category data sets, including the category, sample number, feature number and number of positive samples and negative samples. For example, the caesarian section classification dataset, Indian liver patient dataset (ILPD), glass identification dataset, user knowledge modeling dataset, car dataset and QCM sensor alcohol dataset. The content of data sets covers all aspects of real life, with a wider range and more complex data types. There are both large sample data sets and small sample data sets, and both multi-feature data sets and a small number of feature data sets.

4.3. Performance Metrics

The evaluation indicators used in numerical experiments include the classification accuracy (ACC), sensitivity, precision, specificity and Matthews correlation coefficient (MCC). Sensitivity refers to the ability of the model to identify positive examples. Precision indicates samples with positive prediction results. These are positive cases. The specificity measurement model is the ability to identify negative examples. The measurement range of MCC is $[-1,1]$, and the other is $[0,1]$. The larger the evaluation indicator is, the better the performance of the model under this indicator.

For multi-class datasets, the corresponding concerned data categories are taken as positive categories and other categories are taken as negative categories. The data sets were calculated to obtain the values of relevant evaluation indicators, such as the accuracy (ACC), sensitivity, precision, specificity and Matthews correlation coefficient (MCC).

Standard classification indicators, such as the accuracy (ACC), sensitivity, precision, specificity and Matthews correlation coefficient (MCC), were used in the experiment. According to [25,26,27], the true positive (TP) is the number of positive instances of correct classification, the false negative (FN) is the number of positive instances of incorrect classification, the true negative (TN) is the number of negative instances of correct classification and the false positive (FP) is the number of positive instances of incorrect classification. The basic configuration matrix of TP, FN, TN and FP is shown in

Table 3. The Basic Confusion Matrix.

Basic Confusion Matrix		Predicted Class
		Positive	Negative
Actual Class	Positive	True Positive (TP)	False Negative (FN)
	Negative	False Positive (FP)	True Negative (TN)

.

By referring to the papers [25,26,27], this paper lists the evaluation indicators of accuracy (ACC), sensitivity, precision, specificity and Matthews correlation coefficient (MCC) as follows:

$$ACC=\frac{TP+TN}{TP+TN+FN+FP}$$

(11)

$$Precision=\frac{TP}{TP+FP}$$

(12)

$$Sensitivity=\frac{TP}{TP+FN}$$

(13)

$$Specificity=\frac{TN}{FP+TN}$$

(14)

$$MCC=\frac{TP·TN-FP·FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}$$

(15)

The above evaluation indicators can comprehensively evaluate the performance of the proposed model.

4.4. Baseline Methods

In order to verify the hybrid SCA FKNN model, a large number of data experiments were conducted to verify the effectiveness of the proposed method. Firstly, in the first part, this paper tested two classes of data sets, and made a numerical comparison of nine metaheuristic-based algorithms (LSCA [21], SCA [12], PSO [28], SSA [29], SA [30], BA [31], CGSCA [32], mSCA [33] and CESCA [34]), fixing their $M=2$ and $K=3$ to verify the advantages of execution data results, respectively. In order to ensure the fairness of the numerical experiment, the experiment was repeated five times on the same machine. Based on the repeated experiment, the average value and standard deviation of the model were analyzed. Each experiment included five cross validation results. The average value of five cross validations was taken for performance evaluation.

4.5. Experimental Design

All numerical experiments were calculated by MATLAB 2017. All experiments were conducted on the same equipment, and 8 GB ram Intel Core i5 (Intel, Santa Clara, CA, USA) equipped with windows 11 (Microsoft, Redmond, WA, USA) was used as a workstation to avoid the impact of experimental hardware during the simulation process.

5. Results and Discussion

5.1. Numerical Results for Two-Classes Datasets

For the following three types of two-class datasets, the hybrid SCA FKNN method proposed in this paper will be compared and analyzed with eight metaheuristic algorithms. The superior performance of hybrid SCA FKNN in the evaluation metrics fully shows the advantages of the proposed method, and further verifies the effectiveness of the method.

5.1.1. Experimental Results on the Bupa Dataset

The numerical results of the hybrid SCA FKNN compared with other models in the Bupa dataset are shown in

Table 4. Results of the hybrid FKNN and comparison models on the Bupa dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.7799	0.7015	0.6412	0.8791	0.4728
	std	0.0143	0.0284	0.0299	0.0283	0.0303
LSCA-FKNN	avg	0.6232	0.6674	0.5687	0.8393	0.2465
	std	0.0199	0.0338	0.0175	0.0177	0.0367
SCA-FKNN	avg	0.6175	0.5494	0.4645	0.7968	0.2946
	std	0.0383	0.0439	0.0487	0.0178	0.0546
PSO-FKNN	avg	0.6686	0.6531	0.4851	0.8047	0.3105
	std	0.0266	0.0344	0.0379	0.0231	0.0472
BA-FKNN	avg	0.6056	0.5600	0.4693	0.7121	0.1920
	std	0.0292	0.0479	0.0423	0.0292	0.0694
SSA-FKNN	avg	0.6377	0.5862	0.5667	0.6923	0.2601
	std	0.0586	0.0069	0.1233	0.0327	0.0895
SA-FKNN	avg	0.6721	0.6444	0.4511	0.8087	0.3131
	std	0.0142	0.0340	0.0128	0.0415	0.0279
CGSCA-FKNN	avg	0.6600	0.6486	0.4711	0.7981	0.2939
	std	0.0193	0.0356	0.0195	0.0308	0.0382

. This paper carried out 10 repeated numerical experiments, and the average value and standard deviation of the 10 repeated experiments are listed in Table 4. It can be seen that the hybrid SCA FKNN model proposed in this paper achieves the best results among the four evaluation indicators. The hybrid SCA FKNN model proposed in this paper observes better results on ACC, which are approximately 8.4–25.1% higher than the comparison models. Although the standard deviation of LSCA-FKNN is lower than that of hybrid SCA FKNN in most cases, the numerical results of hybrid SCA FKNN are significantly better than LSCA FKNN in terms of average evaluation index values.

In order to show the overall benchmarking analysis results of each model, Figure 5 draws a bar graph of the performance of each model, draws the average value of 10 repeated experiments of each model and adds the standard deviation of repeated experiments as the error line. Figure 5 is a visual display of Table 4. As shown in Figure 5, except for precision, good results have been achieved in sensitivity, specificity and MCC. It can be clearly seen from the figure that the hybrid SCA FKNN method has better numerical results and stronger stability.

5.1.2. Experimental Results on the Hepatitis Dataset

As reflected in the above table on the Bupa dataset,

Table 5. Results of the hybrid-FKNN and comparison models on the hepatitis dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.9465	0.3638	0.4072	0.9392	0.4342
	std	0.0569	0.0937	0.0845	0.0312	0.0945
LSCA-FKNN	avg	0.8191	0.4566	0.3760	0.9192	0.3276
	std	0.0296	0.0874	0.0676	0.0303	0.0648
SCA-FKNN	avg	0.8051	0.4236	0.3512	0.9167	0.3009
	std	0.0323	0.1061	0.0745	0.0301	0.0875
PSO-FKNN	avg	0.7742	0.4641	0.3376	0.9217	0.3118
	std	0.0378	0.0969	0.0727	0.0230	0.0788
BA-FKNN	avg	0.8172	0.4333	0.4692	0.8305	0.3593
	std	0.0233	0.0702	0.0903	0.0267	0.0812
SSA-FKNN	avg	0.8750	0.2975	0.3333	0.9333	0.3846
	std	0.0432	0.0379	0.1925	0.0087	0.0098
SA-FKNN	avg	0.8076	0.4115	0.3280	0.9097	0.2771
	std	0.0242	0.0913	0.0751	0.0266	0.0797
CGSCA-FKNN	avg	0.8033	0.5533	0.3944	0.9067	0.3744
	std	0.0199	0.0740	0.0666	0.0266	0.0582

shows the benchmarking results between the hepatitis dataset and other model methods. It can be seen that the hybrid SCA FKNN model proposed in this paper achieves the best results among the three evaluation indicators.In terms of sensitivity and precision, the hybrid SCA FKNN performs worse than BA FKNN and CGSCA FKNN, but significantly better than their numerical results in terms of ACC, specificity and MCC. The hybrid SCA FKNN model proposed in this paper observes better results on ACC, which are approximately 15.2–19.6% higher than the comparison models.

Similarly, in order to further visualize the comparison of the five types of evaluation indicators, Figure 6 shows the performance of each model more intuitively. For this data set, the data size of the positive samples is small, at only 32 data, so it has a great impact on the sensitivity and MCC. In general, the hybrid SCA FKNN model has competitive advantages.

ACC is an index used to describe the accuracy of the model. The higher the value of ACC, the better the prediction result of the model. For the current hepatitis dataset, Table 5 is obtained according to the comparison with the numerical results in paper [21]. As shown in Table 5 and Figure 6, the hybrid-FKNN model proposed in this paper obtains better results in ACC, which are approximately 15.2–19.6% higher than the comparison model. However, its variance is high and its stability is poor. The data results are greatly affected by random data sampling.

5.1.3. Experimental Results on the SPECT Dataset

Similarly,

Table 6. Results of the hybrid-FKNN and comparison models on the SPECT dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.8936	0.8620	0.7157	0.5220	0.4436
	std	0.0195	0.0845	0.0610	0.0227	0.0605
LSCA-FKNN	avg	0.7593	0.8538	0.8730	0.4094	0.2588
	std	0.0191	0.0187	0.0357	0.0561	0.0656
SCA-FKNN	avg	0.7297	0.7953	0.8601	0.3427	0.1759
	std	0.0449	0.0283	0.0278	0.0707	0.0718
PSO-FKNN	avg	0.7615	0.8405	0.8541	0.3866	0.2079
	std	0.0351	0.0216	0.0308	0.0637	0.0975
BA-FKNN	avg	0.7585	0.7098	0.9270	0.1049	0.0259
	std	0.0287	0.0064	0.0164	0.0493	0.0702
SSA-FKNN	avg	0.6471	0.5455	0.8571	0.4443	0.3228
	std	0.0899	0.0951	0.0825	0.0673	0.0943
SA-FKNN	avg	0.7658	0.8195	0.8920	0.2891	0.1031
	std	0.0164	0.0176	0.0234	0.1013	0.0637
CGSCA-FKNN	avg	0.7546	0.8497	0.8330	0.4308	0.2034
	std	0.0229	0.0140	0.0238	0.0587	0.0709

shows the benchmarking results of the SPECT dataset and other model methods on the evaluation indicators of ACC, sensitivity, precision and MCC. In terms of sensitivity and precision, the hybrid SCA FKNN performs worse than BA FKNN and LSCA FKNN. BA FKNN has a high sensitivity and low specificity, and there is a significant difference in its ability to recognize positive cases and negative cases. However, in terms of ACC, specification and MCC, it is significantly better than their numerical results.

Figure 7 vividly shows the performance of various models on different evaluation indicators. It can be seen from the figure that the sensitivity, specificity and MCC indicators of the model vary greatly, which may be due to the small number of negative samples, at only 55 negative samples. Cross validation has a great impact on the sensitivity and MCC. It can be seen from the figure that the hybrid SCA FKNN method is quite competitive in obtaining numerical results on SPECT data sets.

For the current SPECT dataset, Table 6 is also obtained by the hybrid SCA FKNN. As shown in Table 6, the hybrid-FKNN model obtains better results in ACC, which are approximately 12.1–14.3% higher than the comparison models. As shown in Figure 7, except for sensitivity and precision, the best results were also achieved in specificity and MCC. However, satisfactory results have been achieved in sensitivity and precision. The numerical results are quite competitive.

In order to further verify the effectiveness of the proposed method, the standard is attached in Appendix A, which concerns the experimental results obtained in hepatitis, Bupa and SPECT datasets under the conditions of different maximum cycle test times and different cross validation numbers for researchers’ reference.

5.2. Numerical Results for Multi-Classes Datasets

For the above seven different application scenarios and different types of data sets, the following data results are obtained in this paper. As shown in

Table 7. Results of the hybrid-FKNN on multi-class datasets.

Datasets	Metric	ACC	Precision	Sensitive	Specificity	MCC
Caesarian section classification dataset	Avg	0.7026	0.7197	0.8336	0.7049	0.4694
	Std	0.0901	0.0500	0.0160	0.0927	0.0747
Indian Liver Patient Dataset (ILPD)	Avg	0.7953	0.7876	0.8255	0.3788	0.1621
	Std	0.0342	0.0379	0.0853	0.0541	0.0437
Glass Identification Dataset	Avg	0.7827	0.9016	0.9526	0.8347	0.4734
	Std	0.0355	0.0501	0.0376	0.0751	0.1037
User Knowledge Modeling Dataset	Avg	0.8606	0.9545	0.9709	0.9185	0.9042
	Std	0.0267	0.0408	0.0163	0.0864	0.0319
Breast Tissue Dataset	Avg	0.6554	0.9667	0.8883	0.9770	0.6969
	Std	0.0727	0.0577	0.0459	0.1443	0.1163
Car Dataset	Avg	0.8807	0.9188	0.9973	0.9823	0.8312
	Std	0.0916	0.0612	0.0716	0.0982	0.1616
QCM sensor Alcohol Dataset	Avg	0.9043	0.8501	0.8568	0.8477	0.8562
	Std	0.0939	0.0838	0.0719	0.0973	0.1008

, for multi-class datasets, the data prediction accuracy is between 0.65–0.90, and the hybrid-SCA FKNN method can still achieve good results. It has good data prediction results for datasets with multiple or few characteristics, and multiple or few samples. This method is more adaptable.

In order to better demonstrate the effectiveness of the method proposed in this paper, this paper compares the numerical results of the above seven different data sets calculated by the LSCA FKNN [21], SCA FKNN [12], PSO FKNN [28], BA FKNN [31] and SSA FKNN [29] methods. The relevant numerical comparison results are shown in

Table 8. Results of the hybrid-FKNN and comparison models on the caesarian section classification dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.7026	0.7197	0.8336	0.7049	0.4694
	std	0.0901	0.0500	0.0160	0.0927	0.0747
LSCA-FKNN	avg	0.6677	0.8148	0.5741	0.7095	0.3923
	std	0.0955	0.1197	0.0986	0.0965	0.1295
SCA-FKNN	avg	0.6667	0.6766	0.6349	0.7333	0.3626
	std	0.0722	0.2384	0.0755	0.0882	0.0414
PSO-FKNN	avg	0.6675	0.8194	0.7505	0.7250	0.4045
	std	0.0701	0.1138	0.1220	0.0992	0.0939
BA-FKNN	avg	0.5625	0.5361	0.7424	0.2500	0.2655
	std	0.0625	0.0804	0.0957	0.0443	0.0995
SSA-FKNN	avg	0.5775	0.6528	0.5370	0.5952	0.2381
	std	0.0523	0.0241	0.0656	0.0591	0.0817

,

Table 9. Results of the hybrid-FKNN and comparison models on the Indian liver patient dataset (ILPD). (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.7953	0.7876	0.8255	0.3788	0.1621
	std	0.0342	0.0379	0.0853	0.0541	0.0437
LSCA-FKNN	avg	0.6912	0.6961	0.9856	0.1078	0.0712
	std	0.0191	0.0123	0.0220	0.0309	0.0257
SCA-FKNN	avg	0.6455	0.7440	0.8882	0.2058	0.1038
	std	0.0222	0.0732	0.0964	0.0821	0.0674
PSO-FKNN	avg	0.7173	0.7187	0.9967	0.0963	0.0953
	std	0.0139	0.0162	0.0058	0.0107	0.0641
BA-FKNN	avg	0.7108	0.7209	0.9534	0.0865	0.0532
	std	0.0085	0.0076	0.0034	0.0012	0.0125
SSA-FKNN	avg	0.7092	0.7215	0.9695	0.0502	0.0644
	std	0.0028	0.0140	0.0277	0.0457	0.0112

,

Table 10. Results of the hybrid-FKNN and comparison models on the glass dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.7827	0.9016	0.9526	0.8347	0.4734
	std	0.0355	0.0501	0.0376	0.0751	0.1037
LSCA-FKNN	avg	0.6589	0.5961	0.6759	0.7618	0.4242
	std	0.0355	0.2470	0.1530	0.0546	0.1972
SCA-FKNN	avg	0.6654	0.6078	0.8614	0.7028	0.5339
	std	0.0355	0.0453	0.0558	0.0337	0.0394
PSO-FKNN	avg	0.6047	0.6429	0.6923	0.7727	0.4587
	std	0.0968	0.0415	0.0994	0.0646	0.0950
BA-FKNN	avg	0.5349	0.3333	0.5000	0.6429	0.3187
	std	0.1005	0.0907	0.0874	0.0707	0.0587
SSA-FKNN	avg	0.7209	0.7778	0.9091	0.6875	0.6209
	std	0.0880	0.0128	0.1905	0.1168	0.2040

,

Table 11. Results of the hybrid-FKNN and comparison models on the user modeling dataset hamdi tolga dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.8606	0.9545	0.9709	0.9185	0.9042
	std	0.0267	0.0408	0.0163	0.0864	0.0319
LSCA-FKNN	avg	0.7901	0.9286	0.9286	0.9808	0.9093
	std	0.0317	0.0299	0.0469	0.0127	0.0388
SCA-FKNN	avg	0.7977	0.9107	0.9639	0.9678	0.9143
	std	0.0744	0.0233	0.0313	0.0138	0.0400
PSO-FKNN	avg	0.7713	0.8860	0.9434	0.9444	0.8876
	std	0.0681	0.0218	0.0246	0.0059	0.0395
BA-FKNN	avg	0.5179	0.6324	0.5404	0.8189	0.3828
	std	0.0890	0.1101	0.3625	0.1877	0.2242
SSA-FKNN	avg	0.8025	0.9437	0.7593	0.9541	0.7018
	std	0.0377	0.0150	0.0590	0.0153	0.0629

,

Table 12. Results of the hybrid-FKNN and comparison models on the breast tissues dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivty	Specificity	MCC
Hybrid SCA-FKNN	avg	0.6554	0.9667	0.8883	0.9770	0.6969
	std	0.0727	0.0577	0.0459	0.1443	0.1163
LSCA-FKNN	avg	0.5397	0.8667	0.6389	0.9333	0.6154
	std	0.0727	0.2309	0.1273	0.1155	0.2682
SCA-FKNN	avg	0.5373	0.7500	0.8167	0.9024	0.6471
	std	0.0550	0.0012	0.1243	0.0117	0.0937
PSO-FKNN	avg	0.5238	0.6583	0.8333	0.7500	0.5677
	std	0.0991	0.0366	0.0787	0.0605	0.0879
BA-FKNN	avg	0.4928	0.5843	0.6875	0.6742	0.4731
	std	0.0727	0.0473	0.0887	0.0751	0.0949
SSA-FKNN	avg	0.6667	0.9487	0.8196	0.6667	0.5657
	std	0.0825	0.0888	0.0239	0.0774	0.0865

,

Table 13. Resultsof the hybrid-FKNN and comparison models on the car dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.8807	0.9188	0.9973	0.9823	0.8312
	std	0.0916	0.0612	0.0716	0.0982	0.1616
LSCA-FKNN	avg	0.8691	0.9450	0.9467	0.8388	0.7922
	std	0.0161	0.0372	0.0289	0.1106	0.0454
SCA-FKNN	avg	0.8168	0.8789	0.9559	0.9428	0.7818
	std	0.0134	0.0095	0.0057	0.0271	0.0110
PSO-FKNN	avg	0.7645	0.7718	0.9977	0.3179	0.4601
	std	0.0159	0.0964	0.0434	0.0530	0.0064
BA-FKNN	avg	0.7107	0.7342	0.9468	0.6591	0.6596
	std	0.0859	0.1090	0.0350	0.1100	0.1226
SSA-FKNN	avg	0.8618	0.9461	0.8816	0.8307	0.6653
	std	0.0854	0.0939	0.0269	0.0819	0.0945

and

Table 14. Results of the hybrid-FKNN and comparison models on the QCM sensor alcohol dataset. (Bold indicates the best in comparison method).

Algorithm	Metric	ACC	Precision	Sensitivity	Specificity	MCC
Hybrid SCA-FKNN	avg	0.9043	0.8501	0.8568	0.8477	0.8562
	std	0.0939	0.0838	0.0719	0.0973	0.1008
LSCA-FKNN	avg	0.7600	0.7917	0.8327	0.8189	0.7269
	std	0.0367	0.0908	0.0823	0.0925	0.1415
SCA-FKNN	avg	0.7702	0.7333	0.7333	0.8841	0.7479
	std	0.0693	0.0887	0.1082	0.0275	0.1270
PSO-FKNN	avg	0.4400	0.5095	0.6656	0.6794	0.2208
	std	0.0400	0.0744	0.0672	0.0531	0.0580
BA-FKNN	avg	0.3067	0.6567	0.2611	0.7804	0.3568
	std	0.0231	0.0333	0.0674	0.0756	0.0632
SSA-FKNN	avg	0.8133	0.6640	0.7714	0.8682	0.6117
	std	0.1007	0.0856	0.0960	0.0304	0.1126

. From the five evaluation indicators, the hybrid SCA FKNN method proposed in this paper achieved good numerical results under seven datasets.

For the current caesarian section classification dataset, Table 8 is also obtained from hybrid SCA FKNN. As shown in Table 8, the hybrid SCA FKNN model achieved better results in ACC, sensitivity and MCC, which were approximately 5.2–21.7% higher than the comparison model in ACC.

For the Indian liver patient dataset (ILPD), Table 9 shows the data results obtained by the hybrid SCA FKNN method and other numerical models. As shown in Table 9, the hybrid SCA FKNN model achieved better results in ACC, precision, specificity and MCC, which were approximately 10.9–23.2% higher than the comparison model in ACC.

For the glass dataset, Table 10 shows the data results obtained by the hybrid SCA FKNN method and other numerical models. As shown in Table 10, the hybrid SCA FKNN model achieved better results in ACC, precision, sensitivity specificity and MCC, which were approximately 8.6–46.3% higher than the comparison model in ACC.

For the user modeling dataset hamdi tolga dataset, Table 11 shows the data results obtained by the hybrid SCA FKNN method and other numerical models. As shown in Table 11, the hybrid SCA FKNN model achieved better results in ACC, precision, sensitivity and specificity, which were approximately 7.2–66.2% higher than the comparison model in ACC.

For the breast tissues dataset, Table 12 shows the data results obtained by the hybrid SCA FKNN method and other numerical models. As shown in Table 12, the hybrid SCA FKNN model achieved better results in precision, sensitivity specificity and MCC, which were slightly lower than SSA-FKNN and higher than the comparison model in ACC.

For the car dataset, Table 13 shows the data results obtained by the hybrid SCA FKNN method and other numerical models. As shown in Table 13, the hybrid SCA FKNN model achieved better results in ACC, specificity and MCC, which were approximately 1.3–8.6% higher than the comparison model in ACC.

For the QCM sensor alcohol dataset, Table 14 shows the data results obtained by the hybrid SCA FKNN method and other numerical models. As shown in Table 14, the hybrid SCA FKNN model achieved better results in ACC, precision, sensitivity and MCC, which were twice as large as the results of PSO FKNN and BA FKNN in ACC.

In order to better verify the method proposed in this paper, this paper analyzed the operation process of the five data sets. The change in best fitness can analyze the convergence of the data running process. If the best fitness does not change during the cycle, the data prediction results do not change in general. If a few cycles are used, the best fitness will not change, indicating that the method has a faster convergence speed.

From Figure 8, it can be seen that the optimal fitness will start to be in a stable state at approximately 12–23 iterations; that is, in the next cycle, the data of the training set will not be optimized, and the method will be in a convergent state. As shown in Figure 8, after the first iteration, the best fitness of the current training set will be obtained, and the current best fitness will not be equal to 0. The value of this best fitness gradually stabilizes with the increase in the number of iterations, but it does not start from 0. For the car dataset, QCM sensor alcohol dataset and glass identification dataset, the best fitness converges quickly and converges to a stable value at approximately the 10th iteration. For the ILPD and the breast tissue dataset, it also converges to a stable value at approximately the 15th iteration.

With the increase in the number of iterations, the best fitness gradually increases on the car dataset. When the number of iterations reaches 15, the best fitness gradually stabilizes to 0.8543. With the update and optimization of the car training dataset, the data prediction result of the car test set reaches 0.8514. For the QCM sensor alcohol dataset, when the number of iterations reaches 10, the optimal fitness reaches a stable state at 0.8400, and the data prediction result of the QCM sensor alcohol test set reaches 0.8520. For the glass identification dataset, when the number of iterations reaches 12, the optimal fitness reaches a stable state at 0.7256, and the data prediction result of the glass identification test set reaches 0.7364. The classification accuracies of the above three data sets in the cycle process, best fitness and test data sets are roughly the same, which shows that this method can find convergent data points in the simulation process effectively and obtain more accurate results.

For the ILPD, when the number of iterations reaches 20, the optimal fitness reaches a stable state at 0.7256, and the data prediction result of the ILPD test set reaches 0.6609. For the breast tissue dataset, when the number of iterations reaches 15, the optimal fitness reaches a stable state at 0.7256, and the data prediction result of the breast tissue test set reaches 0.6475. For the above two datasets, although the accuracy of the method is slightly lower than that of best fitness, the method achieves better results in precision and stability.

According to Figure 8, it can be clearly seen that the method proposed in this paper has a good convergence effect and achieves the purpose of prediction results when fewer cycles are required.

For further experiments, and to compare the results of LSCA-FKNN and SCA-FKNN for all 10 datasets, this paper will run Wilcoxon signed rank test hybrid SCA FKNN. The null hypothesis is that there is no difference between the accuracy of the two classifiers. As shown in the accuracy results of this paper, we reject the null hypothesis (significance level 0.05), and accept that the two classifiers have significantly different accuracies. This result confirms the advantages of hybrid SCA FKNN.

6. Conclusions

The hybrid SCA FKNN algorithm is proposed in this paper based on the improved SCA algorithm combined with reverse learning and an FKNN classifier. It is a further combination of the swarm intelligence algorithm and classifier. This method is a multi-strategy hybrid algorithm that further optimizes the sine cosine algorithm, makes it easier to jump out of local convergence and obtains more accurate numerical solutions. In the process of implementing hybrid SCA into FKNN, this paper mainly uses the FKNN classifier to calculate the prediction accuracy by cross validation as the current best fitness to iteratively optimize the training dataset in order to obtain a more accurate classification. This way, the training set population can be optimized until the training set cannot be optimized any more and the numerical value converges, which can greatly improve the the accuracy of the numerical results. After comparing the numerical results of the hybrid SCA FKNN method with five other methods in 10 data sets, and through the Wilcoxon signed rank test with SCA-FKNN and LSCA-FKNN, the numerical results were significantly improved.

In the next step, we will further consider logic mining to improve the method [27,35,36,37,38] and integrate multiple patterns to optimize swarm intelligence algorithms and obtain a more efficient method. In the process of using FKNN to calculate fitness, this method repeatedly calculates the location distance, requires a lot of numerical calculations and takes a large amount of time. The next step is to consider a more efficient classifier to reduce the time cost. In the next step, we will further optimize the model in combination with spiritual Gaussian mutation [39] to improve the accuracy.