RNA Sequences-Based Diagnosis of Parkinson’s Disease Using Various Feature Selection Methods and Machine Learning

Abstract

Parkinson’s disease is a neurodegenerative disease that is associated with genetic and environmental factors. However, the genes causing this degeneration have not been determined, and no reported cure exists for this disease. Recently, studies have been conducted to classify diseases with RNA-seq data using machine learning, and accurate diagnosis of diseases using machine learning is becoming an important task. In this study, we focus on how various feature selection methods can improve the performance of machine learning for accurate diagnosis of Parkinson’s disease. In addition, we analyzed the performance metrics and computational costs of running the model with and without various feature selection methods. Experiments were conducted using RNA sequencing—a technique that analyzes the transcription profiling of organisms using next-generation sequencing. Genetic algorithms (GA), information gain (IG), and wolf search algorithm (WSA) were employed as feature selection methods. Machine learning algorithms—extreme gradient boosting (XGBoost), deep neural network (DNN), support vector machine (SVM), and decision tree (DT)—were used as classifiers. Further, the model was evaluated using performance indicators, such as accuracy, precision, recall, $F_1$ score, and receiver operating characteristic (ROC) curve. For XGBoost and DNN, feature selection methods based on GA, IG, and WSA improved the performance of machine learning by 10.00% and 38.18%, respectively. For SVM and DT, performance was improved by 0.91% and 7.27%, respectively, with feature selection methods based on IG and WSA. The results demonstrate that various feature selection methods improve the performance of machine learning when classifying Parkinson’s disease using RNA-seq data.

1. Introduction

Parkinson’s disease is a representative neurodegenerative disease that is associated with numerous genetic and environmental factors. However, the genes affecting it are still unclear [1,2,3]. Currently, there is no underlying treatment for Parkinson’s disease, and treatment is ineffective if the symptoms worsen. Therefore, an early diagnosis and treatment of Parkinson’s disease is vital [4].

Research has been conducted to classify cancer [5,6,7,8] and other diseases [9,10,11] using machine learning. For classifying Parkinson’s disease, many recent studies have been conducted using various types of data, such as magnetic resonance imaging (MRI), electroencephalograms (EEG), voice signals, breathing signals, and gene expression data [12,13,14,15,16,17,18]. In particular, the development of next-generation sequencing (NGS) technology enables the definition of the transcriptomic landscape for cancer and other diseases, which aids in patient diagnosis [19,20].

Many studies have conducted a metaheuristic-based feature selection method to improve the performance of machine learning. Metaheuristic algorithms are recognized as robust problem solvers to solve a variety of problems of different types—genetic algorithm [21,22], particle swarm optimization [23], heat transfer search optimization [24], starling murmuration optimizer [25,26], wolf search algorithm [27], and grey wolf optimization [28,29] are some of the successful metaheuristic algorithms, which can be applied to feature selection for machine learning.

The performance of feature selection methods is generally related with machine learning models such as extreme gradient boost (XGBoost), multilayer perceptron (MLP), deep neural networks (DNN), support vector machine (SVM), and decision trees (DT). In particular, tree-based ensemble models such as XGBoost showed outstanding performance on prediction and classification problems [30,31,32,33]. For example, according to R. Sarkhani Benemaran et al. [34], the combination of particle swarm optimization and the XGBoost model showed good performance.

Research on classifying diseases by feature selection of RNA sequence (RNA-seq) has been steadily presented in the literature [35,36,37]. This study used data from 29 patients with Parkinson’s and 44 RNA-seq recordings without Parkinson’s [38]. RNA-seq data were classified using machine learning using various feature selection methods. There was only one paper [39] that used the same data as this study. In this previous work, a deep neural network called DEGnet was proposed to predict only UR and DR genes in Parkinson’s disease, and the classification problem, which is investigated in this study, was not addressed. The machine learning models used in this study include extreme gradient boost (XGBoost), deep neural networks (DNN), support vector machine (SVM), and decision trees (DTs). Cross-validation was conducted to evaluate the performance of the model. Furthermore, the model was evaluated using receiver operating characteristic (ROC) curves. In this study, genetic algorithms, information gain, and wolf search algorithms are examined as feature selection methods. There have been no previous studies comparing the performances of these three feature selection methods with RNA-seq data.

The main contribution of this paper can be summarized in three points as follows:

• To verify the performance of feature selection when classifying diseases using machine learning with RNA-seq data.
• To investigate how various feature selection methods affect the performance of machine learning.
• To compare the performance of machine learning according to various feature selection methods (genetic algorithms, information gain, wolf search algorithms).

The rest of the paper is organized as follows: Section 2 describes the background associated with this study; Section 3 describes the proposed various feature selection methods and the hyperparameters of each machine learning algorithm; Section 4 shows the score of each machine learning algorithm and ROC curve; Section 5 summarizes and discusses the results for the various feature selection methods and machine learning classification; and finally, Section 6 concludes this study.

2. Background

2.1. RNA Sequencing (RNA-seq)

RNA-seq is a technique used for analyzing the transcriptome profile of organisms using next-generation sequencing [40]. With the growth of NGS technology, RNA-seq has been used for gene expression analysis, co-expression network analysis, alternative specifying analysis, simple generic sequence repeat analysis, and pathway analysis since 2005 [41].

RNA-seq utilizes deep-base technologies. The process is as follows: An adapter is first mounted on both ends of RNA and converted into a cDNA fragment library [42]. After basing the created sequencing library, the parts of the genome of the DNA base sequence that are similar are analyzed [43]. RNA-seq was obtained through this process. In contrast with conventional research methods that focus on genes and tissues, RNA-seq allows all genetic information to be read and analyzed simultaneously.

2.2. Dataset Used for This Study

We searched for Parkinson’s disease data on the National Center for Biotechnology Information (NCBI), an online site with diverse biological data. As a result, we obtained RNA-seq data that [44] measured RNA expression levels in the brain tissue of 73 people (both with and without Parkinson’s disease). These data showed 17,580 protein-coding genes identified by RNA sequencing [38].

Table 1. Summarized statistics on RNA-seq by A. Dumitriu et al. [38].

Analysis	Description	Control	PD	t-Test p
RNA-seq	Number of samples	44	29	-
	Age at death, years (range)	70.00 (46–97)	77.55 (64–95)	4.6 × 10−3
	PMI, hours (range)	14.36 (2–32)	11.14 (1–31)	1.7 × 10−1
	RIN (range)	7.85 (6.0–9.1)	7.07 (5.8–8.5)	5.9 × 10−5

shows a statistical summary of the number of samples, age of death, post-mortem interval (PMI), and RNA integrity number (RIN) of control and with Parkinson’s disease. Through RNA sequencing, A. Dumitriu et al. confirmed that they identified 17,580 protein-coding genes, and 1095 (6.2%) of them were significantly different (FDR p-value < 0.05), but out of 3558 proteins, only 166 protein code genes (0.94%) were significant in false detection rate (FDR) [38].

2.3. Feature Selection

Feature selection methods can be categorized into three types: filter, wrapper, and embedded. The filter method uses predetermined criterions, such as Pearson correlation and mutual information between features, to select the optimal features [45]. In this case, a subset of the selected features was input into the classification algorithm. This method has the advantage of calculating quickly because it is independent of the classification algorithm [46]. The wrapper method uses a specific classification algorithm for feature selection. A subset of features optimized for a particular classification algorithm was selected by generating and evaluating various subsets [47,48]. Although it is more accurate than the filter method, the computation is complicated because it requires executions of classification algorithms on subsets of all features. Embedded methods combine feature selection and learning procedures into a single process to reduce computational time [49]. Therefore, in contrast to other methods, feature selection and classification algorithms cannot be separated [50]. This study used genetic algorithm (GA), information gain (IG), and wolf search algorithm (WSA) as feature selection methods.

2.4. Genetic Algorithm (GA)

GAs are optimization techniques developed by Holland, which mimic Darwin’s theory of evolution [51]. The main mechanism of GAs is that individuals who are more suited to their environment have a better chance of survival. The good genes of surviving individuals are passed on to the next generation, eventually leaving only the good genes [52].

GAs perform tasks such as initialization, selection, crossover, and mutation. Here, initialization usually generates a population randomly. The size of a population can vary depending on the problem. As the generation progresses, the selection of a portion of the population is required to create new individuals [53]. Roulette wheel selection and tournament selection are well known. The selected group creates a new offspring through crossover and mutation. Previous research shows that having better parents results in better offspring [54]. This process is repeated until the termination condition is met [55].

2.5. Information Gain (IG)

Information gain is an entropy-based feature evaluation method, widely used in the field of machine learning [56]. Information gain calculates the reduction in entropy from the transformation of a dataset. The equation for calculating information gain is as follows:

$$IG\left(T, a\right)=H\left(T\right)-H\left(T|a\right)$$

(1)

where $T$ is a random variable, and $H\left(T|a\right)$ is the entropy of $T$ given the attribute $a$ [57]. It can be used for feature selection by evaluating the information gain of each variable in the context of the target variable.

2.6. Wolf Search Algorithm (WSA)

Wolf search algorithm is a new bioinspired heuristic optimization algorithm that is based on the way wolves search for food and survive by avoiding their enemies [27]. Each wolf in WSA hunts independently by remembering its characteristics. Unlike ants that communicate using pheromones, wolves in WSA do not need communication. Consequently, WSA can reduce search time by omitting communication. WSA can be explained in three rules based on the wolf’s hunting behavior [58,59]:

• Each wolf has a fixed radius, which represents the visual area in 2D. Coverage is the area of the circle by the fixed radius. Each wolf can detect a companion appearing within its coverage, and the distance that the wolf can move is smaller than the radius.
• The quality of the wolf is represented using the objective function. Wolves try to move to better terrain. If there are one or more better positions, the wolf will choose the best terrain to inhabit among the given options.
• Wolves can sense enemies and escape to random positions away from threats and beyond their fixed radius.

WSA obtains an optimal solution by gathering multiple leaders rather than by searching in a single direction based on these basic rules [60].

2.7. Machine Learning Algorithms for Classification

Machine learning can be categorized into supervised, unsupervised, semi-supervised, and reinforcement learning [61]. Supervised learning predicts labels when new data are input. Unsupervised learning is a method of determining clusters based only on input data that are not labeled. Semi-supervised learning is a kind of machine learning that uses labeled and unlabeled data [62]. In this case, models are learned using training data including labels. Reinforcement learning is an optimization approach for machine learning that interacts with the environment to learn optimal behavior [63]. In this study, we used supervised learning to classify Parkinson’s disease using features selected by GAs. We selected XGBoost, DNN, SVM, and DT as classifiers.

XGBoost is a tree-based algorithm that was proposed by T. Chen et al. [44]. XGBoost uses the rules of decision tree to classify the data into leaves. It uses the leaves’ scores to calculate the final prediction. The objective function of XGBoost is to minimize the regularized objective by adding a regularization term as follows:

$$Obj\left(\theta \right)=L\left(\theta \right)-\mathsf{\Omega }\left(\theta \right)$$

(2)

where $L$ is the loss function, and $\mathsf{\Omega }$ is the regularization term used to avoid overfitting by controlling the model’s complexity. $\mathsf{\Omega }\left(\theta \right)$ is expressed as follows [44]:

$$\mathsf{\Omega }\left(\theta \right)=\gamma T+\frac{1}{2}\lambda {\Vert w\Vert }^2$$

(3)

where $T$ is the number of leaf nodes, and $w$ is the weight of each leaf. $\gamma$ and $\lambda$ are regularization parameters [64,65,66,67]. XGBoost is created using boosting techniques. Gradient-boosting techniques use residuals from existing models to generate new models for final prediction [68]. Gradient descent algorithms are used to add new models, and thus, they exhibit excellent efficiency and high prediction accuracy [69].

DNNs have emerged through technological development of artificial neural networks. A DNN is an artificial neural network consisting of an input, output, and two or more hidden layers. Based on DNNs, various algorithms, such as convolution neural network (CNN) and recurrent neural network (RNN), have been developed, and they have shown excellent performance in various fields, such as computer vision and natural language processing [70].

SVM is a classification and regression machine learning model used in various fields, including pattern recognition [71] and image classification [72]. The principle of SVM is to generate one or more feature vectors for the classification and decision boundaries between two classes [73]. SVM is an optimal hyperplane-based mathematical learning scheme that transforms the original training data into multidimensional spaces to perform classification and construct hyperplanes in high dimensions [74]. SVM is basically a linear classifier that classifies linearly separable data. However, real-world data may not be linearly separated. To overcome this, kernel tricks are used. Thus, the performance of SVM depends on selecting the appropriate kernel [75]. The linear kernel is the simplest kernel used for SVM. The expression of the linear kernel function [76] is as follows:

$$K\left(x_i, x_j\right)=\langle x_i,x_j\rangle$$

(4)

where $x_i$ and $x_j$ denote data points, and $K$ denotes kernel function.

A DT is a diagram consisting of root, branch, and leaf nodes as one of the training learning models. The branches of the tree represent decisions, occurrences, or reactions [77]. The root node is the highest of all the nodes. Each node represents a shape, each branch represents a decision rule, and the leaf node represents the result [78].

3. Method

3.1. Feature Selection Using Genetic Algorithms

In this study, we performed feature selection using GAs. Figure 1 shows a flowchart of the GA. According to Haupt [79], diversity decreases if the population size is less than 30, and if the population size is more than 120, the GA converges more slowly. Accordingly, the population size was set to 100 so that diversity was guaranteed without slowing down the convergence speed. In addition, the number of features was set to 25, 50, 75, and 100 out of 17,850 for comparison. In order to select the best number of features, the experiment was conducted with various numbers of features. We executed experiments using two fitness functions. The fitness functions are represented in the following [80,81]:

$$w\times \left|\overline{\rho \left(X,Y\right)}\right|+{\left|{\rho }^{\prime }\left(X,P\right)\right|}^{-1}$$

(5)

$$\frac{k\times {\rho }^{\prime }\left(X,P\right)}{\sqrt{k+k\times \left(k-1\right)\times \rho \left(X,Y\right)}}$$

(6)

where $w$ is a constant as an intercorrelation weight factor, which was set to 2 in this experiment, and $k$ is the number of features. $\left|\overline{\rho \left(X,Y\right)}\right|$ is the correlation value between features $X$ and $Y$, and ${\left|{\rho }^{\prime }\left(X,P\right)\right|}^{-1}$ is the correlation value between feature $X$ and label $P$. The genetic algorithm using Equation (5) (GA1) converges as the value decreases, and the genetic algorithm using Equation (6) (GA2) converges as the value increases. In each generation, the top 50% were selected as parents based on fitness. For a pair of parent chromosomes, one-point crossover with a 100% probability was applied, and two offspring were created. After the crossover operation, a 10% probability of bit-flip mutation was applied. The bottom 50% of chromosomes were replaced with the newly created offspring.

Figure 2 and Figure 3 shows the best fitness of each generation of genetic algorithms using Equation (5) (GA1) and genetic algorithms using Equation (6) (GA2). The appropriate number of generations was determined based on this result. Both algorithms converge without improvement after the 80th generation. Therefore, we set the number of generations to 100. The whole experimental results of machine learning models according to the number of features selected by GA1 and GA2 are provided in Appendix A.

3.2. Feature Selection Using Information Gain

This method measures attributes’ significance by the information gain calculated to the target class. The combination of this attribute evaluator with the ranking method of search is applied to the RNA-seq dataset [82]. We conducted an information gain experiment using Weka and selected 25, 50, 75, and 100 attributes as input values for machine learning.

3.3. Feature Selection Using Wolf Search Algorithm

WSA was performed using Weka in this study. The attribute evaluator adopted for the experiment was a correlation-based feature selection (CFS) algorithm. CFS finds a subset of features highly correlated with the target class and lowly correlated with the other features [81,83]. Wolf search algorithm was performed to find the best feature set based on this CFS evaluator. Both the number of iterations and the population size of WSA were set to 100. The start points for search were set to 1 and 10.

3.4. Machine Learning Model Implementation

Table 2. Hyperparameters of the classification algorithm used in this experiment.

Model	Parameter	Explanation	Value
XGBoost	n_estimators	Specify the number of gradient boosted trees	100
	learning_rate	Determine the percentage of improvement in learning rates	0.1
	gamma	Minimum loss reduction required to make a further partition on a leaf node of the tree	0
	max_depth	Maximum tree depth for base learners	6
	objective	Specify corresponding learning objectives	binary
SVM	kernel	Sets the type of kernel that the algorithm uses to classify	linear

lists the parameters of XGBoost and SVM used in the experiment. In XGBoost used in this study, the number of gradient boost trees was set to 100, and the learning rate was set to 0.1. The gamma of the used XGBoost was set to 0, and the maximum depth was set to 6. To conduct a binary classification (Parkinson’s and control), the learning objective was set to binary. For SVM, the C was set to 1, and $\gamma$ was set to 1/(number of features · variance of input values). To classify the two classes, SVM used linear kernel.

Table 3. DNN structures generated using data selected by feature selection.

Layer	Output Shape	Param #
Input	51, 100	-
Linear-1	51, 128	12,928
Dropout-2	51, 128	0
Linear-3	51, 128	16,512
Linear-4	51, 64	8256
Linear-5	51, 64	4160
Linear-6	51, 32	2090
Linear-7	51, 32	1056
Linear-8	51, 16	528
Linear-9	51, 1	17

lists the layer, output shape, and number of parameters of the DNN used in the experiment. It consisted of one input layer, seven hidden layers, and one output layer. In addition, a dropout layer was used to prevent the overfitting of the model. For the activation function of all layers except the output layer, the rectified linear unit (ReLU) function was used to prevent gradient vanishing. The activation function of the output layer was set to the sigmoid function usually used in binary classification. The expressions for the ReLU and sigmoid functions are as follows:

$$f\left(x\right)=x^{+}\equiv \max \left(0,x\right)=\left\{\begin{array}{c} 0 \left(x<0\right)\\ x \left(x\ge 0\right)\end{array}\right.$$

(7)

$$Sigmoid\left(x\right)=\sigma \left(x\right)=\frac{1}{1+\exp \left(-x\right)}$$

(8)

where $x$ represents the input value of the network.

When training the model, the stochastic gradient descent method (SGD) was chosen as the optimizer, and the learning speed and momentum were set to 0.001 and 0.9, respectively. The expression for SGD is as follows [84]:

$$\begin{gathered} v_{t+1}=\mu \ast v_t+g_{t+1} \\ {p}_{t+1}=p_t-lr\ast v_{t+1} \end{gathered}$$

(9)

where $p$ represents the parameters, $g$ represents the gradient, $v$ represents the velocity, $lr$ represents the learning rate, and μ represents the momentum.

Because the activation function of the output layer is a sigmoid function, the loss function is set to binary cross entropy loss (BCELoss). The BCELoss equation is as follows:

$$l_n=-\frac{1}{M}{\displaystyle \sum }_{m=1}^M\left[y_m\times \log \left(h_{\theta }\left(x_m\right)\right)+(1-y_m\times \log \left(1-h_{\theta }\left(x_m\right)\right)\right]$$

(10)

where $M$ represents the number of training sessions, $y_m$ represents the actual label at the $m$th learning session, $x_m$ represents the input value at the $m$th learning session, and $h_{\theta }$ represents the weight $\theta$ of the model. The epoch, the number of times the model was learned, was set to 250. The batch size of the DNN, which refers to the number of training sets used in one iteration, was set to be the same as the size of the training set. That is, just one batch was used for DNN.

In the case of DT, the quality of the split was evaluated using Gini impurity [85] as a criterion, and the best split was chosen. The minimum number of samples required to split an internal node was set to 2.

The four machine learning models used in the experiment were implemented using Python. The sklearn and PyTorch libraries were used to create the model, and the source code is attached to https://github.com/JingeunKim/PDclassification (accessed on 14 February 2023). All machine learning models were learned and tested on systems with Apple M1 Max GPU, Apple M1 Max @ 3.22 GHz, and 32 GB memory.

3.5. Model Evaluation

K-fold cross-validation was used for model evaluation. K-fold cross-validation divides the dataset into K sets and uses the K-1 set as the training set and the other as the test set. The average accuracy was obtained by repeating this process K times. In this study, five was used as the K value.

The $accuracy$ (Equation (11)), $precision$ (Equation (12)), $recall$ (Equation (13)), and $F_1 score$ (Equation (14)) based on the confusion matrix were used to evaluate the performance of the machine learning models. The expressions for the $accuracy$, $precision$, $recall$, and $F_1 score$ are as follows:

$$Accuracy=\frac{TP+TN}{TP+FP+FN+TN}$$

(11)

$$Precision=\frac{TP}{TP+FP}$$

(12)

$$Recall=\frac{TP}{TP+FN}$$

(13)

$$F_1 score=2\times \frac{Precision\times Recall}{Precision+Recall}$$

(14)

Here, TP, TN, FP, and FN are true positives, true negatives, false positives, and false negatives, respectively. In addition, the model was evaluated using an ROC curve based on $sensitivity$ (Equation (15)) and $specificity$ (Equation (16)). The equations for $sensitivity$ and $specificity$ are as follows:

$$Sensitivity=\frac{TP}{TP+FN}$$

(15)

$$Specificity=\frac{TN}{TN+FP}$$

(16)

4. Results

We performed feature selection based on the three feature selection methods based on genetic algorithm, information gain, and wolf search algorithm. Then, the performance of machine learning using the three feature selection methods were compared with that of machine learning not using the feature selection. In this experiment, to obtain a feature subset of 100 of 17,850 features, runs of GA1, GA2, and IG took 16,221 s, 16,056 s, and 1 s, respectively. A run of WSA took 41,273 s to obtain a subset of 73 features out of 17,850 features. Preprocessing, such as normalization or standardization, for using the selected feature as an input value of machine learning is omitted. In this study, fivefold cross-validation was used to evaluate the machine learning models. That is, 4/5, which is 80%, of the total data were used as a training set, and 1/5, which is 20%, of data were used as a test set for each fold. When classifying Parkinson’s disease using the dataset used in this study, it was confirmed that data preprocessing reduced the accuracy of machine learning.

Figure 4 shows the performance of machine learning on the four feature selection methods used in the experiment. In the case of XGBoost, the four evaluation criteria increased when performing feature selection, excluding GA2, compared to when not performing feature selection. In the case of DNN, the accuracy of non-feature selection was approximately 47%, showing lower accuracy than that of XGBoost, but the accuracy of feature selection through GA1 and WSA was approximately 85%, showing the highest performance among the machine learning. In the case of SVM and DT, it can be observed that the model’s performance decreases by performing feature selection using genetic algorithm and WSA.

GA1 showed performance improvements in XGBoost and DNN, GA2 showed performance improvements in DNN, IG showed performance improvements in all machine learning, and WSA showed performance improvements in XGBoost, DNN, and DT.

Figure 5, Figure 6, Figure 7 and Figure 8 show the performance of the four machine learning models using the ROC curves. Each figure shows the mean ROC curve of the five folds and the ROC curve for each fold. In these figures, for each machine learning model, the result with the best feature selection method and the result without feature selection were compared.

As shown in Figure 5 and Figure 6, it can be observed that the performance of machine learning using feature selection improved compared to the case where feature selection was not used. For example, the average AUC of XGBoost learned through feature selection was approximately 87%, and for the DNNs, the average AUC was approximately 89%. Similarly, Figure 7 and Figure 8 show that, for SVM and DT, the model’s AUC increases when using feature selection based on IG.

We also analyzed whether the results with the best feature selection for each machine learning model were statistically significant using t-tests.

Table 4. Best Feature Selection Method by Machine Learning Model, Number of Features, and Machine Learning Performance by Fold.

Feature Selection Method	Number of Features	ML	Fold	Accuracy	Precision	Recall	${\mathit{F}}_1$	$\mathit{p}$ Value between the Feature Selection and Non-Feature Selection
IG	50	XGBoost	1	86.36	89.28	86.36	86.10	2.57 × 10−4
			2	90.90	90.83	90.83	90.83
			3	95.45	95.45	95.83	95.44
			4	81.81	83.03	80.83	81.19
			5	81.81	80.35	80.35	80.35
GA1	100	DNN	1	90.90	92.85	90.00	90.59	3.14 × 10−13
			2	81.81	81.66	81.66	81.66
			3	86.36	86.75	85.83	86.10
			4	90.90	92.85	90.00	90.59
			5	77.27	77.27	77.50	77.22
IG	50	SVM	1	72.72	79.46	77.27	76.84	4.38 × 10−1
			2	90.90	90.83	90.83	90.83
			3	72.72	73.33	73.33	72.72
			4	72.72	79.52	75.83	76.03
			5	90.90	90.17	90.17	90.17
WSA	73	DT	1	68.18	68.33	68.18	68.11	2.41 × 10−4
			2	95.45	95.45	95.83	95.44
			3	77.27	77.27	77.50	77.22
			4	81.81	83.03	80.83	81.19
			5	63.63	62.50	63.39	62.39

shows the result. In this table, the p-value of the t-test between the result of each feature selection and that of non-feature selection was also included for statistical analysis. For each machine learning model, the best among the results of five folds is marked in bold. According to the t-test, for XGBoost, DNN, and DT, the performances of the models were improved by using feature selection methods (0.05 > $p$). However, for SVM, there was no significant difference between the result of feature selection and that of non-feature selection, with a $p$ value of 0.43 (0.05 < $p$).

5. Discussion

Table 5. Comparison of accuracy, precision, recall, $F_1$ score, and time cost of machine learning for feature selection methods and non-feature selection.

	Number of Features	Model	Accuracy (%)	Precision	Recall	${\mathit{F}}_1$	Time Cost (s)
Non-feature selection	17,850	XGBoost	77.27	76.60	76.44	77.03	26.45
		DNN	47.27	50.00	32.06	23.64	16.74
		SVM	80.91	80.20	80.43	82.52	38.28
		DT	70.00	68.62	68.59	70.51	11.44
GA1	75	XGBoost	80.91	80.94	80.53	81.36	1.02
	75	DNN	85.45	86.28	85.00	85.24	10.84
	50	SVM	79.09	79.69	78.92	78.64	0.07
	50	DT	69.09	68.68	68.12	67.77	0.09
GA2	50	XGBoost	74.55	76.18	74.56	73.73	1.12
	75	DNN	84.55	86.69	83.67	83.72	4.86
	50	SVM	73.64	74.67	73.68	72.63	0.35
	50	DT	68.18	69.33	68.70	67.66	0.08
IG	50	XGBoost	87.27	87.79	86.84	86.79	0.98
	100	DNN	81.82	83.96	80.83	81.16	6.56
	50	SVM	81.82	82.67	81.49	81.32	0.06
	50	DT	77.27	78.90	77.37	76.43	0.08
WSA	73	XGBoost	86.36	89.04	84.95	85.46	0.77
	73	DNN	85.45	87.18	84.50	84.89	4.61
	8358	SVM	80.00	81.55	79.10	79.33	5.85
	73	DT	77.27	77.32	77.15	76.88	0.09

The best result among the experimented feature selection methods for each machine learning method is marked in bold.

lists the performance of XGBoost, DNN, SVM, and DT. The table contains the number of features that showed the best result with the corresponding feature selection method for each machine learning model. The best result among the experimented feature selection methods for each machine learning method is marked in bold. Time cost in the table represents the time spent to train and test the model. In this table, XGBoost shows higher performance and shorter computing time when performing feature selection based on GA1, IG, and WSA than when not performing feature selection. With IG, four performance indicators increased by at least 9.76%, and the CPU time required to train the model was reduced by 26.98 times. DNN showed a low accuracy of 47.27% when executed without selecting features. However, when machine learning used feature selection through GA1 and WSA, the performance was improved to 85.45%. We observed a 37.18% increase in precision, an indicator of how well Parkinson’s disease patients were predicted, when machine learning was performed with WSA. In addition, the CPU time could be reduced by four times when using feature selection. 8 × 10⁻³.

In the case of SVM and DT, four performance indicators decreased when feature selection was performed with GA1 and GA2. However, performance increased when feature selections based on IG and WSA were performed. In addition, measured CPU time decreased when using feature selection methods. Therefore, using the appropriate feature selection methods can improve the performance of machine learning and reduce computing time.

6. Conclusions

In this study, we compared various feature selection methods for machine learning algorithms to classify Parkinson’s disease better. We experimentally validated that various feature selection methods improved the performance of machine learning when classifying Parkinson’s disease using RNA-seq data. For XGBoost and DNN, feature selection methods based on GA1, IG, and WSA improved the performance when classifying Parkinson’s disease. The accuracy of classification by XGBoost increased by 10.00%, from 77.27% to 87.27%, with the IG-based feature selection method. Similarly, the accuracy of DNN also increased with feature selection methods based on WSA and GA1 by 38.18%, from 47.27% to 85.45%. In SVM and DT, feature selection methods based on IG and WSA showed good performance compared to GA-based feature selection. With IG-based feature selection, the classification accuracy of SVM increased from 80.91% to 81.82%, while that of DT increased from 70.00% to 77.27%. The results of these experiments confirmed that feature selection is effective for classification of Parkinson’s disease using RNA-seq data. In addition, the computational costs were reduced for all machine learning models tested when using feature selection.

The experimental results demonstrate that appropriate feature selection could improve the performance of machine learning and reduce computational costs. For future research, it will be necessary to develop an optimization algorithm that can shorten the time cost for feature selection and further improve the performance of machine learning.