A Dynamic Tasking-Based Evolutionary Algorithm for Bi-Objective Feature Selection

Abstract

Feature selection in classification is a complex optimization problem that cannot be solved in polynomial time. Bi-objective feature selection, aiming to minimize both selected features and classification errors, is challenging due to the conflict between objectives, while one of the most effective ways to tackle this is to use multi-objective evolutionary algorithms. However, very few of these have ever reflected an evolutionary multi-tasking framework, despite the implicit parallelism offered by the population-based search characteristic. In this paper, a dynamic multi-tasking-based multi-objective evolutionary algorithm (termed DTEA) is proposed for handling bi-objective feature selection in classification, which is not only suitable for datasets with relatively lower dimensionality of features, but is also suitable for datasets with relatively higher dimensionality of features. The role and influence of multi-tasking on multi-objective evolutionary feature selection were studied, and a dynamic tasking mechanism is proposed to self-adaptively assign multiple evolutionary search tasks by intermittently analyzing the population behaviors. The efficacy of DTEA is tested on 20 classification datasets and compared with seven state-of-the-art evolutionary algorithms. A component contribution analysis was also conducted by comparing DTEA with its three variants. The empirical results show that the dynamic-tasking mechanism works efficiently and enables DTEA to outperform other algorithms on most datasets in terms of both optimization and classification.

1. Introduction

Feature selection is an essential skill for machine learning [1] and multi-objective feature selection, especially for high-dimensional datasets, and has become a hot research topic in the field of optimization. For instance, bi-objective feature selection in classification [2] normally seeks both the smallest number of selected features and the minimum value of classification error rates. Due to a potential contradiction, we need to find a compromise and balance between these two objectives—one of the most effective ways is by using multi-objective evolutionary algorithms (MOEAs) [3,4,5,6,7,8]. Compared with traditional search mechanisms, an evolutionary algorithm (EA) [9], like many other meta heuristics [10], has no need for specific domain knowledge or problem presumption. Moreover, its population-based searching characteristics are naturally suitable for finding various nondominated feasible solutions with an environmental selection trade-off [11].

Nowadays, after long-term development, MOEAs have generated a variety of mainstream frameworks [12]. From the point of view of environmental selection, there are MOEAs based on dominance relationships [13,14,15,16,17,18], decomposition strategies [19,20,21,22], performance indicators [23,24,25,26], and distance measurements [27,28,29,30]. In addition, from the perspective of hybrid applications, there are also MOEAs that are combined with surrogate models [31,32,33], cooperative coevolutionary mechanisms [34,35,36], and evolutionary multi-tasking frameworks [37,38,39], which can solve multiple tasks simultaneously via knowledge transfer. Finally, from the viewpoint of addressing optimization problems, there are MOEAs used for high-dimensional objective space [40,41,42,43,44,45], complex Pareto fronts [46,47,48,49], multi-modal environments [50,51,52,53], and large-scale decision variables [54,55,56].

There are numerous other kinds of excellent MOEAs [57,58,59,60] that have been proposed around the world, many of which are used for real-world applications, such as intrusion detection in networks [61], efficient sensing in wireless sensor networks [62], control of building systems [63], menu planning in schools [64] and control of hybrid electric vehicle charging systems [65].

Recently, large-scale multi-objective optimization problems (LSMOPs) [66] have attracted a lot of attention, which often challenge MOEAs to their limits due to the tremendous difficulty of searching the decision space. Some MOEAs adopt multi-tasking frameworks to solve LSMOPs, such as the multi-variation multi-factorial evolutionary algorithm [67] that conducts a search on both the original space of the LSMOP and multiple simplified spaces constructed in a multi-variation manner concurrently; the evolutionary multitasking algorithm [68] that self-adaptively adjusts the search range of each individual by estimating the population evolving trend; and the multi-objective multitasking algorithm [69] with subspace distribution alignment and decision variable transfer strategies that can promote positive information transfer. Such methods have shown some success in multi-tasking when addressing LSMOPs.

Moreover, the high-dimensional feature selection problem [70] (i.e., also an LSMOP) is even more challenging to address due to the sparse and discrete search space, not to mention the complicated relationship or interactions between real-world features in a dataset. It should also be noted that the term “large-scale decision space” in the field of LSMOPs actually means high-dimensional features in this paper, because the decision or search space of a feature selection problem is precisely the feature space. However, to the best of our knowledge, there are still very few studies that have focused on applying an evolutionary multi-tasking framework to the high-dimensional multi-objective feature selection problem, in spite of the promising combination of learning techniques and evolutionary algorithms [71]. Therefore, in this work, we focus on studying how a dynamic evolutionary multi-tasking framework behaves in bi-objective feature selection optimization, whether in high-dimensional or low-dimensional datasets, with an attempt to explore a universal mechanism for self-adaptively distributing multiple evolutionary search tasks within a population. Consequently, a dynamic tasking-based evolutionary algorithm (termed DTEA) is proposed, with its major contributions given as follows:

• For the first time, a dynamic multi-tasking framework is proposed to self-adaptively assign evolutionary search tasks to different parts of a population for tackling bi-objective feature selection, especially in datasets with relatively higher dimensionality of features. Unlike other approaches, the proposed multi-tasking system aims at adjusting the search behaviors on tasks to fit the real-time evolutionary demands, without any reformation of the original optimization problem, hence reducing the risk of negative transfer.
• In the early evolutionary stage, the algorithm decides whether or not to add a backward search task in order to boost the convergence of the whole population, while part of the solutions will be reassigned and redeployed. Then, with the help of cross-cultural genetic transfer, the backward and main task solutions will mutually pull each other, thereby approaching to meet in objective space and, finally, merging into a single task again.
• At a later evolutionary stage, the algorithm decides whether to add a region of interest (ROI) search task, if the convergence has come to maturity and the ROI can be estimated. In that case, the computational resources will be rearranged to concentrate on searching only the ROI area by reassigning part of the solutions to the ROI task which will be intermittently updated every 10 generations according to the current nondominated front of the main task.

The remainder of this paper is organized as follows: First, background knowledge, including the feature selection problem to be solved and related work, is introduced in Section 2. Then, the general framework and the essential components of the proposed DTEA are illustrated in Section 3. Following this, the experimental setups are described in Section 4, while the empirical studies undertaken are presented in Section 5. Finally, the conclusions and a consideration of future work are provided in Section 6.

2. Background

2.1. Bi-Objective Feature Selection Problem

The multi-objective optimization problem (MOP) [72] with a total of M objectives and D decision variables is normally defined as follows:

$$\begin{array}{c}minimize\mathit{F}\left(\mathit{x}\right)={(f_1\left(\mathit{x}\right),f_2\left(\mathit{x}\right),\dots ,f_M\left(\mathit{x}\right))}^T\hfill \\ subjectto\mathit{x}\in \Omega ,\hfill \end{array}$$

(1)

where $\mathit{x}=(x_1,x_2,\dots ,x_D)$ is a decision vector in the D-dimensional decision space $\Omega$, $\mathit{F}\left(\mathit{x}\right)$ is an objective vector of $\mathit{x}$ in the M-dimensional objective space, and $f_i\left(\mathit{x}\right)$ is the ith objective function of $\mathit{x}$.

With respect to the bi-objective feature selection problem in this paper, two objectives are minimized: the number of selected features and the resultant classification error of the selected features, both of which can be transformed to ratios between 0 and 1. Drawn from the D-dimensional feature space, solution $\mathit{x}$ is a discrete vector that contains either 0 or 1, while $x_i=1$ means selecting the ith feature and $x_i=0$ means not. Thereby, the first objective function $f_1\left(\mathit{x}\right)$ in (1) can be further specified as:

$$f_1\left(\mathit{x}\right)={\sum }_{i=1}^D{x}_i/D,$$

(2)

indicating that, if a decision vector $\mathit{x}$ has selected k features, there will exist k elements in $\mathit{x}$ with $x_i=1$ and all the others ($D-k$ elements) with $x_i=0$, resulting in the objective value in the $f_1$ direction being $k/D$.

Moreover, the second objective function $f_2\left(\mathit{x}\right)$ in (1) denotes the resultant classification error ratio based on the selected features and a preset classifier. Here, we conduct the so-called wrapper-based approach [4,73] for evolutionary feature selection, which directly uses a classifier to evaluate the classification error in real time, during evolution. Theoretically, the classification error can be reduced by selecting more valid features but the computational cost will also be high if too many features are selected.

2.2. Related Works

As previously introduced, although there are some promising works [67,68,69,74,75] applying evolutionary multi-tasking frameworks to solve LSMOPs or high-dimensional single-objective feature selection problems, almost none has focused on the high-dimensional multi-objective feature selection case. MOEAs with multi-tasking as described in [67,68,69,74,75] are only applicable to either the continuous optimization environment or the single-objective optimization case. Recently, a multi-task decomposition-based evolutionary algorithm (MTDEA) [76] was proposed to specifically tackle bi-objective feature selection, which is based on a combination of decomposition approaches and multi-task mechanisms. The most fundamental difference between the above MTDEA and the proposed DTEA in this paper is that MTDEA focuses on the adjustment of global idea points in decomposition-based approaches, while DTEA focus on studying the dynamic and adaptive control of multi-task mechanisms during evolution. Moreover, DTEA adopts totally different environmental selection methods and evolutionary search tasks from MTDEA, which aims to better balance the diversity and convergence factors in searching the Pareto optimal solutions.

There are many other MOEAs [5,77,78] that do not adopt multi-tasking but are specifically designed for tackling the bi-objective feature selection problem, which are described here. For instance, an MOEA based on a duplication analysis mechanism (DAEA) is proposed in [79] to tackle feature selection, where the duplicated solutions in both the objective and decision space are adaptively filtered before environmental selection. DAEA handles relatively low-dimensional datasets rather well, but its ability has not yet been demonstrated on relatively much higher-dimensional datasets. In addition, an MOEA specially designed for solving the sparse subset selection problems (SparseEA) is proposed in [80] to accelerate the convergence and search speed in the large-scale decision space, which can adaptively initialize and reproduce the population according to the fitness contribution of each feature evaluated at the beginning of the evolution. SparseEA handles relatively high-dimensional datasets quite well, but it may encounter early maturity and falls into local optima on relatively much lower-dimensional datasets. Furthermore, an MOEA based on segmented initialization and modified reproduction methods (SIOM) is proposed in [81] to provide a universal plug-in unit for traditional MOEAs to adapt to bi-objective feature selection optimization scenarios. Like DAEA, the search ability of SIOM has not yet been fully verified on relatively much higher-dimensional datasets.

3. Proposed Algorithm

In this section, we first elaborate the general framework of the proposed algorithm, and then discuss the processes of reproduction and selection. Finally, the details of utilizing two essential components, i.e., the backward and ROI search tasks, are  illustrated.

3.1. General Framework

The general principles of the proposed dynamic multi-tasking mechanism can be roughly divided into two stages: the early evolutionary stage where main and backward search tasks coexist, shown in Figure 1a, and the later evolutionary stage where the main and ROI search tasks coexist, shown in Figure 1b.

In Figure 1, in the beginning, an initial population is generated and entirely assigned to the main task. After 10 generations (an empirical setting), the convergence state of the population is analyzed to decide whether or not to reassign part of the solutions to the backward search task and to redeploy those solutions at the front end of the $f_1$ direction (using the algorithms further detailed in Section 3.3). As shown in Figure 1a, the squares denoting the backward task solutions are understood to lie in front of the dots denoting the main task solutions, thereby simulating a backward or forward search towards each other. Moreover, owing to so-called cross-cultural information transfer [37], the main and backward tasks mutually approach at an increasingly faster speed, and finally merge into a single main task again. In this way, the convergence speed is greatly increased, with the help of the backward task searching the feature subsets of much smaller sizes.

After merging, when the convergence state of the population comes to maturity, part of the solutions will be reassigned to the ROI search task and to explore only a restricted area. As shown in Figure 1b, the so-called ROI area is estimated by the current nondominated front (i.e., from $1/D$ to $3/D$ in the $f_1$ direction) of the main task solutions, but with $1/D$ larger in the $f_1$ direction (i.e., extending the right boundary of ROI from $3/D$ to $4/D$). Moreover, the ROI area is updated intermittently every 10 generations as the nondominated front of the main task solutions changes. Thus, by concentrating the computational resources on the most promising ROI area, the ROI task is intended to help find more nondominated solutions reserved for the final classification test in a more efficient way.

The general framework of the proposed algorithm DTEA is shown in Algorithm 1, with necessary annotations following directly after the codes. More detailed explanations are given as follows: In Line 2, the cross-cultural mating rate is a parameter to control the random possibility of genetic exchanges between solutions of two different tasks (later used in Line 8), as described in [37].

In Line 4, there is only a main search task at the beginning, defined as type 1 in the task pool. In Line 6, the archived population is used as a comparison with the current one so as to track the convergence rate for the past 10 generations. Moreover, the $Reproduce$, $Select$, $AddBack$ and $AddROI$ methods in Lines 8, 9, 14 and 16 are detailed in Algorithms 2, 3, 4 and 6, respectively. It should also be noted that all the multiple tasks are conducted within a single population and cooperate through the cross-cultural genetic exchanges among solutions, keeping the population size always the same.

3.2. Reproduction and Selection

Algorithm 2 illustrates the reproduction process that not only outputs new offspring but also plays a key role in the cross-cultural information transfer between different tasks. In fact, the multi-tasking reproduction mechanism used in DTEA is similar to that in MO-MFEA [37], except for adopting the modified crossover mechanism in DAEA [79] that reduces invalid crossover to some extent. Moreover, the cross-cultural mating rate is preset to 0.5 in DTEA (Line 2 of Algorithm 1) in order to obtain a balanced information transfer ratio, with no attempt to find its optimal value.

Algorithm 1  $DTEA(N,D)$

Input: population size N, total feature number D; Output: final population $Pop$; 1: $Pop\leftarrow$ randomly sample N initial solutions from the D-dimensional feature space; 2: $R=0.5$; $//$set cross-cultural mating rate 3: $Gen=0$; $//$set generation count 4: $T=(1,1,\dots ,1)$; $//$create N size task pool 5: $ROI=\varnothing$; $//$set an empty ROI object 6: $Arc=Pop$; $//$archive old population 7: while termination criterion not met do 8:     $(Po{p}^{*},T^{*})=Reproduce(Pop,T,R)$; 9:     $Pop=Select(Pop,Po{p}^{*},T,T^{*},ROI)$; 10:     $Gen=Gen+1$; 11:     if $Gen\%10==0$ then 12:         if only one task type in T then 13:            if $Gen==10$ then 14:                $(Pop,T)=AddBack(Pop,T,Arc)$; 15:            else if $ROI==\varnothing$ then 16:                $(ROI,T)=AddROI(Pop,T,Arc)$; 17:            end if 18:         else 19:            if $ROI==\varnothing$ then 20:                merge & update T; $//$see Section 3.3 21:            else 22:                update $ROI$; $//$see Section 3.4 23:            end if 24:         end if 25:         $Arc=Pop$; 26:     end if 27: end while

Algorithm 2  $Reproduce(Pop,T,R)$

Input: current population $Pop$, task pool T, cross-cultural mating rate R; Output: offspring set $Po{p}^{*}$, offspring task pool $T^{*}$; 1: $Par\leftarrow$ randomly choose N pairs of parents from $Pop$ by tournament selection in terms of the associated nondominated fronts and crowding distances; 2: for$i=1,2,\dots ,N$do 3:     $〈a,b〉\leftarrow$ get parent indexes in $Par\left(i\right)$; 4:     if $T\left(a\right)==T\left(b\right)$ then 5:         $Po{p}^{*}\left(i\right)\leftarrow$ get offspring by crossover & mutation; 6:         $T^{*}\left(i\right)=T\left(a\right)$; $//$assign the same task as parents 7:     else 8:         if $rand<R$ then 9:            $Po{p}^{*}\left(i\right)\leftarrow$ repeat Line 5; 10:            $T^{*}\left(i\right)\leftarrow$ randomly assign $T\left(a\right)$ or $T\left(b\right)$; 11:         else 12:            $Po{p}^{*}\left(i\right)\leftarrow$ only do mutation to $Pop\left(a\right)$; 13:            $T^{*}\left(i\right)=T\left(a\right)$; 14:         end if 15:     end if 16: end for

Algorithm 3 demonstrates the environmental selection process that individually and successively selects the fittest solutions for each evolutionary search task. The multi-tasking selection mechanism of DTEA is generally based on that of MO-MFEA [37], but conducts totally different tasks and adopts a matched duplication removal mechanism. As can be seen from Lines 4 and 6, the main and backward tasks only remove the duplicates within themselves, while the ROI task further removes its duplicates within the main task. In this way, the main and backward tasks can evolve more independently so as to stimulate their respective convergence potential, while the ROI task aims at supplying more diversity for the nondominated front obtained by the main task. As for truncation, the main and backward tasks use the same classic nondominated sorting and crowding distance methods [13], while the ROI task uses another selection method, which is explained later in Section 3.4.

Algorithm 3  $Select(Pop,Po{p}^{*},T,T^{*},ROI)$

Input: current population $Pop$, offspring set $Po{p}^{*}$, task pool T, offspring task pool $T^{*}$, region of interest $ROI$; Output: next-generation population $Pop$; 1: $\psi =Pop\bigcup Po{p}^{*}$; 2: $\phi =T\bigcup T^{*}$; 3: ${\psi }^1,{\psi }^2,{\psi }^3\leftarrow$, respectively, get solutions from $\psi$ corresponding to $\phi =1,2,3$; $//$1-main, 2-backward, 3-ROI tasks 4: ${\psi }^1,{\psi }^2,{\psi }^3\leftarrow$ respectively, remove the duplicated decision vector solutions of their own; 5: ${\psi }^1,{\psi }^2\leftarrow$ individually truncate ${\psi }^1,{\psi }^2$ by nondominated sorting and crowding distance; 6: ${\psi }^3\leftarrow$ remove same decision vector solutions with ${\psi }^1$; 7: ${\psi }^3=SelectROI({\psi }^3,ROI)$; $//$see Section 3.4; 8: $Pop={\psi }^1\bigcup {\psi }^2\bigcup {\psi }^3$;

3.3. Backward Search Task

The utilization of the backward search task consists of two steps: judging whether to add the backward task (called by Line 14 of Algorithm 1 and detailed in Algorithm 4), and judging whether to merge it into the main task (called by Line 20 of Algorithm 1 and detailed in Figure 2).

In Algorithm 4, the objective values of the current and archived population in the $f_1$ direction are used to estimate the convergence rate and to judge the adding of the backward task (Lines 1 and 2). If adding, the scale of the backward task is set according to the objective values in the $f_2$ direction (Lines 3∼8). Here, the scale ranges between 0.25 and 0.5 of N and is negatively correlated with the convergence speed in the $f_2$ direction. As shown in Line 9, solutions with the worst nondominated fronts and crowding distances are allocated from the main task to the backward task and are redeployed by Algorithm 5.

Algorithm 4  $AddBack(Pop,T,Arc)$

Input: current population $Pop$, task pool T, archived population $Arc$; Output: redeployed population $Pop$, reassigned tasks T; 1: $F_1^1,F_1^2\leftarrow$ get $f_1$ objective values from $Pop,Arc$, respectively; 2: if  $min\left(F_1^1\right)>mean\left(F_1^2\right)/2$  then 3:     ${\psi }^1$ and ${\psi }^2\leftarrow$ respectively, get nondominated solutions from $Pop$ and $Arc$; 4:     $F_2^1$ and $F_2^2\leftarrow$ respectively, get unique $f_2$ objective values from ${\psi }^1$ and ${\psi }^2$; 5:     $n_1\leftarrow$ number of duplicates between $F_2^1$ and $F_2^2$; 6:     $r_1=n_1/size\left(F_2^1\right)$; $//$$f_2$ duplicate rate 7:     $r_2=sigmoid(r_1\ast 10)/2$; $//$backward task ratio 8:     $n_2=floor(r_2\ast N)$; $//$backward task scale 9:     $index\leftarrow$ sort solutions of $Pop$ in descending order with nondominated fronts as the primary criterion and crowding distances as the secondary criterion; 10:     $T\left(index(1,2,\dots ,n_2)\right)=2$; $//$reassign tasks 11:     $Pop\left(index(1,2,\dots ,n_2)\right)=Redeploy(n_2,D,1/D)$; 12: end if

Algorithm 5  $Redeploy(K,D,H)$

Input: redeployed solution number K, total feature number D, distribution ratio H; Output: redeployed solution set S; 1: for$i=1,2,\dots ,K$do 2:     for $j=1,2,\dots ,D$ do 3:         if $rand<H$ then 4:            $x_j^i=1$; $//$select the jth feature 5:         else 6:            $x_j^i=0$; $//$not select the jth feature 7:         end if 8:     end for 9:     $S\left(i\right)={\mathit{x}}^i$; 10: end for

In Algorithm 5, a required number of solutions are redeployed in objective space by re-initializing individuals with a smaller number of selected features. As reported in [81], the distribution of an initial population in the bi-objective feature selection case can be roughly controlled by the distribution ratio used in Line 3. In other words, the distribution ratio reflects the mathematical expectation of the selected features in each individual. Here, it is set to $1/D$, meaning that the individual is expected to select around one feature or a bit more, causing the backward task solutions redeployed at the front end of the objective space to simulate a backward search towards the back end.

The final stage of the backward search task is to be merged into the main task, which is determined by the mechanism shown in Figure 2, where the squares denote the objective vectors of the main task and the dots denote those of the backward task. First of all, let us assume the rectangle $A_1$ (with two edges $E_1$ and $E_2$ shown in Figure 2) is made up of the maximum and minimum objective values in both $f_1$ and $f_2$ directions (i.e., the four nodes of a rectangle) for all the backward search task-related solutions, while the same goes for the rectangle $A_2$ (with two edges $E_3$ and $E_4$ shown in Figure 2) for all the main search task-related solutions. By multiplying the two edges of $A_1$ or $A_2$ (i.e., $E_1,E_2$ or $E_3,E_4$), we can obtain the area size of either $A_1$ or $A_2$. Then, let us define another rectangle $A_3$, which denotes the overlapping area of the two dashed rectangles ($A_1$ and $A_2$) in Figure 2. Hence, the upper left and lower right nodes of $A_3$ can be easily obtained by finding the two intersection points of $A_1$ and $A_2$, i.e., the intermediate nodes between $A_1$ and $A_2$ in both $f_1$ and $f_2$ directions. The area size of $A_3$ can be directly calculated by the above obtained upper left and lower right nodes of $A_3$. Therefore, if the area size of $A_3$ is close to that of either $A_1$ or $A_2$, it is indicated that the backward and main tasks are approaching each other closely in the objective space. Moreover, if $A_3$ has become as large as either $A_1$ or $A_2$, the multiple evolutionary tasks will then be merged by updating the task pool shown in Line 20 of Algorithm 1 with all the elements in T reset to 1 (i.e., all turning into the main task).

3.4. ROI Search Task

The utilization of the ROI search task consists of three parts: first, judging whether to add and construct it (called by Line 16 of Algorithm 1, and then detailed in Algorithm 6 and Figure 3); second, intermittent updating of the $ROI$ object (called by Line 22 of Algorithm 1 and detailed in Figure 4); third, the ROI selection method (called by Line 7 in Algorithm 3 and detailed in Algorithm 7).

Algorithm 6  $AddROI(Pop,T,Arc)$

Input: current population $Pop$, task pool T, archived population $Arc$; Output: region of interest $ROI$, reassigned tasks T; 1: ${\psi }^1,{\psi }^2\leftarrow$, respectively, get nondominated solutions from $Pop,Arc$ by nondominated sorting; 2: if ${\psi }^1$ duplicate with ${\psi }^2$ in decision space then 3:     $n_1=N-size\left({\psi }^1\right)$; 4:     $n_2=min(n_1,floor(N/2))$; $//$ROI task scale 5:     $index\leftarrow$ sort solutions of $Pop$ in descending order with nondominated fronts as the primary criterion and crowding distances as the secondary criterion; 6:     $T\left(index(1,2,\dots ,n_2)\right)=3$; $//$reassign tasks 7:     $\phi \leftarrow$ sum element values of each decision vector in ${\psi }^1$; $//$get number of selected features in each nondominated solution 8:     $n_3=max\left(\phi \right)$; $//$get number of selected features in rightmost nondominated solution 9:     first row of $ROI\leftarrow 1,2,\dots ,n_3,n_3+1$; 10:     second row of $ROI$ contains the number of needed solutions corresponding to each element of first row, the calculation of which is shown in Figure 3; 11: end if

Algorithm 7  $SelectROI({\psi }^3,ROI)$

Input: candidate solutions ${\psi }^3$, ROI object $ROI$; Output: reserved solutions ${\psi }^3$; 1: $L\leftarrow$ length of the first row in $ROI$; 2: $S=\varnothing$; $//$used to store selected indexes 3: for  $i=L-1,\dots ,2,1,L$  do 4:     if $ROI(2,i)>0$ then 5:         $inde{x}_1\leftarrow$ get indexes of unselected candidates; 6:         $inde{x}_2\leftarrow$ sort solutions of ${\psi }^3\left(inde{x}_1\right)$ in ascending order with $abs(f_1-ROI(1,i)/D)$ as primary criterion and $f_2$ as secondary criterion; 7:         $S=S\bigcup inde{x}_1\left(inde{x}_2(1,2,\dots ,ROI(2,i))\right)$; 8:     end if 9: end for 10: ${\psi }^3={\psi }^3\left(S\right)$; $//$get final reserved solutions

At first, in Algorithm 6, we construct a 2-dimensional $ROI$ object to help conduct the ROI search task, while the adding criterion is shown in Lines 1 and 2. Specifically, the criterion requires that each nondominated solution of the current population can find a duplicate decision vector in the archived population 10 generation ago, indicating that the convergence has kind of come to maturity. In Lines 3 and 4, the ROI task scale is set according to the number of solutions that are not nondominated, and is restricted under half of the population size so that the main task solutions still hold the majority. Similar to the backward task, only some solutions with the worst nondominated fronts and crowding distances are reassigned to the ROI search task (Line 6). Moreover, the construction of the $ROI$ object is shown in Lines 7∼10 and further illustrated in Figure 3.

Figure 3 demonstrates a topological example of constructing an $ROI$ object with the ROI task scale set to 9, with the $f_1$ value of the rightmost nondominated solution set to $3/D$. Hence, the ROI search boundary is set to $4/D$ (i.e., $1/D$ larger than the nondominated front boundary) for extensive exploration, and the first row of $ROI$ contains all the possible sizes of feature subsets within that boundary (i.e., $〈1,2,3,4〉$). In fact, the so-called $ROI$ object can be treated as a set of ROI subproblems, while each subproblem consists of the number of selected features and the number of needed solutions. The former denotes the selection bias, and the latter denotes how many solutions should be selected. The constructing order of each needed solution is marked inside the circles, starting from right to left, layer-by-layer, but only remaining one at the right end. By counting the number of those circles hanging on each dashed vertical line related to the number of selected features, we obtain the corresponding number of needed solutions for each ROI subproblem (i.e., the second row of $ROI$).

After construction, the scale of the ROI task is fixed, but the structure of the $ROI$ object will be intermittently updated every 10 generations, according to the changes of the current nondominated main task solutions. In Figure 4, a topological example of updating the $ROI$ object inherited from Figure 3 is illustrated, where the ROI task scale is still 9 but the $f_1$ value of the rightmost nondominated solution now changes to $4/D$. Hence, the ROI search boundary is extended to $5/D$, and the first row of $ROI$ is adjusted to $〈1,2,3,4,5〉$. The constructing order in Figure 4 follows a similar principle to that in Figure 3, except for additionally taking the distribution of nondominated main task solutions into account. Specifically, we can regard those nondominated main task solutions as pre-constructed needed solutions, and then reconstruct the needed solutions on that basis, so that the general distribution of nondominated solutions (from both the main and ROI tasks) at each feature subset size will be more uniform.

With the construction or update of the $ROI$ object, the ROI selection method (i.e., the final part of the ROI task utilization) is illustrated in Algorithm 7 (though first called by Line 7 in Algorithm 3). In Algorithm 7, the ROI selection can actually be seen as a bias and sequential subproblem selection process, with a selection order shown in Line 3 (i.e., from the penultimate to the first, plus the last), where each column in $ROI$ can be treated as a subproblem. This means that we start the search from the most promising area (i.e., around the solution with the smallest $f_2$ value) and end at the right boundary of the ROI area for extensive exploration. Moreover, the selection bias of the ROI search task is shown in Lines 5 and 6, where the first criterion is about the number of selected features ($f_1$ values) and the second one is about the classification error ($f_2$ values). In this way, the computational resources will be optimized and focused in order to obtain more diverse nondominated solutions uniformly distributed at each feature subset size in the ROI search area.

4. Experimental Setups

4.1. Test Datasets and Comparison Algorithms

In the experiment, a total of 20 classification datasets are tested, all of which come from real-world data and can be obtained from public databases, such as the UCI machine learning repository [82].

Table 1. Attributes for each classification dataset used as test problems.

No.	Name	Feature	Sample	Class	N	T
1	Wine	13	178	3	50	5000
2	Australian	14	690	2	50	5000
3	Climate	18	540	2	50	5000
4	German	24	1000	2	50	7500
5	Breast	30	569	2	50	7500
6	Ionosphere	34	351	2	50	7500
7	Lung	56	32	3	100	15,000
8	Sonar	60	208	2	100	20,000
9	Hill_Valley	100	606	2	100	25,000
10	Semeion	256	1593	10	100	25,000
11	Madelon	500	2600	2	100	25,000
12	SRBCT	2308	83	4	100	25,000
13	Leukemia1	5327	72	3	100	25,000
14	DLBCL	5469	77	2	100	25,000
15	Brain1	5920	90	5	100	30,000
16	Leukemia2	7070	72	2	100	30,000
17	CNS	7129	60	2	100	30,000
18	Arcene	10,000	200	2	100	30,000
19	Brain2	10,367	50	4	100	30,000
20	Prostate	10,509	102	2	100	30,000

provides the attributes of each test instance, including the numbers of features, samples, and classes, while the population size and the termination criterion (i.e., number of objective function evaluations) run on each dataset is denoted by N and T, respectively. Here, the value of T is set according to the convergence difficulty of each dataset, normally increasing as the total number of features rises, while the value of N is set to 50 for some very small datasets and fixed at 100 for all the other datasets (including the high-dimensional ones) in order to test the search efficiency of the comparison algorithms. Overall, the total feature number ranges from 13 to 10,509 in ascending order, with two or more classes, while the preset population sizes and termination criteria also increase progressively from low-dimensional to high-dimensional datasets.

For comparison, seven state-of-the-art MOEAs are run under the same experimental conditions as the proposed DTEA, including SparseEA [80], DAEA [79], GFMMOEA [49], MOEAHD [47], ARMOEA [83], NSGAII [13] and HypE [23]. Among them, SparseEA and DAEA are two recently published MOEAs specifically designed for bi-objective feature selection, especially for high-dimensional datasets. GFMMOEA is a nondominated sorting-based algorithm that trains a generalized simplex model to dynamically estimate the shape of nondominated fronts. MOEAHD is an improved decomposition-based MOEA that hierarchically adjusts the weight vectors to better tackle complex Pareto fronts. ARMOEA uses an enhanced inverted generational distance indicator to generate a set of reference points that adaptively guides the evolutionary directions. NSGAII and HypE are two of the most classic MOEAs, based, respectively, on nondominated sorting and performance indicators. In addition, three variants of DTEA are also compared to determine the component contributions of the proposed dynamic multi-tasking framework. They are, respectively, the variant without an ROI search task (named DTEA_Back), the variant without a backward search task (named DTEA_ROI), and the variant with only a main search task (named DTEA_Main).

4.2. Performance Indicators and Parameter Settings

Since no single indicator can provide a comprehensive evaluation [84,85], multiple indicators are used in this paper to evaluate the algorithm performance, with hypervolume (HV) [86] as the main indicator and the inverted generation distance (IGD) [87] as a supplement. HV and IGD are two classic and widely used metrics for the evolutionary multi-objective optimization, details about which can be obtained in [26,28,47,79,81]. Moreover, specially for the measurement of feature selection performance, another two metrics, called the minimum classification error (MCE) and the selected feature number (SFN), are adopted, which were also used in [79]. MCE is the minimum classification error obtained by all the solutions, while SFN is the corresponding number of selected features. Generally speaking, smaller IGD, MCE and SFN values represent better performance, which is opposite that for HV. It is worth noting that the Wilcoxon test with a significance level of $5\%$ is also applied to check the performance differences.

The parameter setting for each comparison algorithm is consistent with their own literature. If not specially stated in their literature, the reproduction process will use the same traditional one-point crossover and bit-wise mutation methods. It should be noted that all the algorithms are coded in MATLAB, conducted on the open-source platform PlatEMO [88], and run for 30 times independently with a series of reproducible random seeds. As for classification, each dataset here is randomly divided into the training and test subsets with a proportion of approximately $7/3$, following the stratified split process [79]. Moreover, a KNN ($K=5$) classification model is used on the training set to measure the classification performance during the evolutionary feature selection process, along with a 10-fold cross-validation to avoid feature selection bias [89,90].

5. Results and Analyses

In this section, we first analyze the optimization performances of all the algorithms by the HV and IGD metrics, and then illustrate their solution distributions in the objective space for an intuitive comparison. In addition, the classification performance is further discussed in terms of the MCE and SFN metrics, and finally, the component contributions of DTEA on the backward and ROI search tasks are studied.

5.1. Optimization Performances

Table 2. Mean HV performances of each algorithm on each dataset, with best ones marked in gray, and insignificant differences prefixed by ⋆.

No.	Data	DTEA	SparseEA	DAEA	GFMMOEA	MOEAHD	ARMOEA	NSGAII	HypE
1	Wine	8.78$\times {10}^{-1}$	⋆ 8.74$\times {10}^{-1}$	⋆ 8.76$\times {10}^{-1}$	8.67$\times {10}^{-1}$	⋆ 8.77$\times {10}^{-1}$	⋆ 8.73$\times {10}^{-1}$	⋆ 8.74$\times {10}^{-1}$	⋆ 8.74$\times {10}^{-1}$
		±1.02$\times {10}^{-2}$	±1.13$\times {10}^{-2}$	±1.04$\times {10}^{-2}$	±1.21$\times {10}^{-2}$	±9.82$\times {10}^{-3}$	±1.17$\times {10}^{-2}$	±1.19$\times {10}^{-2}$	±1.14$\times {10}^{-2}$
2	Australian	7.99$\times {10}^{-1}$	7.96$\times {10}^{-1}$	⋆ 7.99$\times {10}^{-1}$	7.93$\times {10}^{-1}$	⋆ 7.99$\times {10}^{-1}$	⋆ 7.98$\times {10}^{-1}$	⋆ 7.99$\times {10}^{-1}$	⋆ 7.98$\times {10}^{-1}$
		±5.48$\times {10}^{-3}$	±5.05$\times {10}^{-3}$	±6.23$\times {10}^{-3}$	±1.14$\times {10}^{-2}$	±4.87$\times {10}^{-3}$	±5.61$\times {10}^{-3}$	±5.74$\times {10}^{-3}$	±5.92$\times {10}^{-3}$
3	Climate	8.90$\times {10}^{-1}$	⋆ 8.89$\times {10}^{-1}$	⋆ 8.90$\times {10}^{-1}$	8.85$\times {10}^{-1}$	⋆ 8.89$\times {10}^{-1}$	⋆ 8.89$\times {10}^{-1}$	⋆ 8.90$\times {10}^{-1}$	⋆ 8.90$\times {10}^{-1}$
		±6.44$\times {10}^{-3}$	±6.15$\times {10}^{-3}$	±6.35$\times {10}^{-3}$	±6.95$\times {10}^{-3}$	±6.91$\times {10}^{-3}$	±6.78$\times {10}^{-3}$	±7.00$\times {10}^{-3}$	±6.70$\times {10}^{-3}$
4	German	7.28$\times {10}^{-1}$	⋆ 7.28$\times {10}^{-1}$	⋆ 7.27$\times {10}^{-1}$	7.18$\times {10}^{-1}$	⋆ 7.27$\times {10}^{-1}$	⋆ 7.26$\times {10}^{-1}$	⋆ 7.28$\times {10}^{-1}$	⋆ 7.27$\times {10}^{-1}$
		±7.83$\times {10}^{-3}$	±6.09$\times {10}^{-3}$	±5.86$\times {10}^{-3}$	±9.67$\times {10}^{-3}$	±9.01$\times {10}^{-3}$	±6.35$\times {10}^{-3}$	±6.17$\times {10}^{-3}$	±6.37$\times {10}^{-3}$
5	Breast	9.40$\times {10}^{-1}$	⋆ 9.38$\times {10}^{-1}$	⋆ 9.39$\times {10}^{-1}$	9.36$\times {10}^{-1}$	⋆ 9.37$\times {10}^{-1}$	⋆ 9.39$\times {10}^{-1}$	⋆ 9.39$\times {10}^{-1}$	9.34$\times {10}^{-1}$
		±4.40$\times {10}^{-3}$	±4.73$\times {10}^{-3}$	±3.09$\times {10}^{-3}$	±8.36$\times {10}^{-3}$	±5.13$\times {10}^{-3}$	±7.06$\times {10}^{-3}$	±6.63$\times {10}^{-3}$	±1.33$\times {10}^{-2}$
6	Ionosphere	8.86$\times {10}^{-1}$	⋆ 8.83$\times {10}^{-1}$	⋆ 8.85$\times {10}^{-1}$	8.73$\times {10}^{-1}$	⋆ 8.86$\times {10}^{-1}$	⋆ 8.85$\times {10}^{-1}$	⋆ 8.83$\times {10}^{-1}$	⋆ 8.85$\times {10}^{-1}$
		±8.75$\times {10}^{-3}$	±1.12$\times {10}^{-2}$	±7.79$\times {10}^{-3}$	±1.04$\times {10}^{-2}$	±8.18$\times {10}^{-3}$	±1.20$\times {10}^{-2}$	±1.06$\times {10}^{-2}$	±9.58$\times {10}^{-3}$
7	Lung	7.38$\times {10}^{-1}$	⋆ 7.29$\times {10}^{-1}$	⋆ 7.32$\times {10}^{-1}$	6.76$\times {10}^{-1}$	6.91$\times {10}^{-1}$	7.11$\times {10}^{-1}$	7.03$\times {10}^{-1}$	⋆ 7.14$\times {10}^{-1}$
		±6.23$\times {10}^{-2}$	±7.02$\times {10}^{-2}$	±5.97$\times {10}^{-2}$	±6.83$\times {10}^{-2}$	±6.50$\times {10}^{-2}$	±6.69$\times {10}^{-2}$	±6.68$\times {10}^{-2}$	±6.45$\times {10}^{-2}$
8	Sonar	8.13$\times {10}^{-1}$	⋆ 8.08$\times {10}^{-1}$	⋆ 8.05$\times {10}^{-1}$	7.89$\times {10}^{-1}$	7.99$\times {10}^{-1}$	7.95$\times {10}^{-1}$	⋆ 8.03$\times {10}^{-1}$	7.89$\times {10}^{-1}$
		±1.96$\times {10}^{-2}$	±2.31$\times {10}^{-2}$	±2.43$\times {10}^{-2}$	±3.13$\times {10}^{-2}$	±1.97$\times {10}^{-2}$	±2.31$\times {10}^{-2}$	±2.21$\times {10}^{-2}$	±3.25$\times {10}^{-2}$
9	Hill_Valley	6.32$\times {10}^{-1}$	⋆ 6.28$\times {10}^{-1}$	⋆ 6.29$\times {10}^{-1}$	5.99$\times {10}^{-1}$	6.26$\times {10}^{-1}$	6.03$\times {10}^{-1}$	6.16$\times {10}^{-1}$	6.23$\times {10}^{-1}$
		±7.98$\times {10}^{-3}$	±8.86$\times {10}^{-3}$	±1.22$\times {10}^{-2}$	±2.10$\times {10}^{-2}$	±9.70$\times {10}^{-3}$	±2.69$\times {10}^{-2}$	±1.91$\times {10}^{-2}$	±1.43$\times {10}^{-2}$
10	Semeion	8.44$\times {10}^{-1}$	8.35$\times {10}^{-1}$	⋆ 8.43$\times {10}^{-1}$	7.11$\times {10}^{-1}$	7.85$\times {10}^{-1}$	7.30$\times {10}^{-1}$	7.39$\times {10}^{-1}$	7.26$\times {10}^{-1}$
		±6.83$\times {10}^{-3}$	±7.82$\times {10}^{-3}$	±5.31$\times {10}^{-3}$	±1.93$\times {10}^{-2}$	±1.52$\times {10}^{-2}$	±1.74$\times {10}^{-2}$	±1.52$\times {10}^{-2}$	±1.11$\times {10}^{-2}$
11	Madelon	9.08$\times {10}^{-1}$	9.06$\times {10}^{-1}$	⋆ 9.08$\times {10}^{-1}$	6.04$\times {10}^{-1}$	6.69$\times {10}^{-1}$	6.15$\times {10}^{-1}$	6.28$\times {10}^{-1}$	6.60$\times {10}^{-1}$
		±3.47$\times {10}^{-3}$	±2.52$\times {10}^{-3}$	±3.26$\times {10}^{-3}$	±4.80$\times {10}^{-2}$	±2.63$\times {10}^{-2}$	±4.75$\times {10}^{-2}$	±3.58$\times {10}^{-2}$	±3.73$\times {10}^{-2}$
12	SRBCT	9.63$\times {10}^{-1}$	⋆ 9.44$\times {10}^{-1}$	9.34$\times {10}^{-1}$	3.23$\times {10}^{-1}$	2.77$\times {10}^{-1}$	3.05$\times {10}^{-1}$	3.33$\times {10}^{-1}$	3.23$\times {10}^{-1}$
		±3.57$\times {10}^{-2}$	±4.44$\times {10}^{-2}$	±4.09$\times {10}^{-2}$	±2.24$\times {10}^{-3}$	±1.81$\times {10}^{-3}$	±2.41$\times {10}^{-3}$	±6.11$\times {10}^{-2}$	±2.24$\times {10}^{-3}$
13	Leukemia1	9.54$\times {10}^{-1}$	⋆ 9.53$\times {10}^{-1}$	⋆ 9.47$\times {10}^{-1}$	5.59$\times {10}^{-1}$	5.48$\times {10}^{-1}$	5.45$\times {10}^{-1}$	5.64$\times {10}^{-1}$	5.53$\times {10}^{-1}$
		±3.49$\times {10}^{-2}$	±3.02$\times {10}^{-2}$	±2.86$\times {10}^{-2}$	±2.20$\times {10}^{-2}$	±1.68$\times {10}^{-2}$	±2.51$\times {10}^{-2}$	±1.53$\times {10}^{-2}$	±2.13$\times {10}^{-2}$
14	DLBCL	9.26$\times {10}^{-1}$	8.97$\times {10}^{-1}$	⋆ 9.05$\times {10}^{-1}$	6.26$\times {10}^{-1}$	6.04$\times {10}^{-1}$	6.07$\times {10}^{-1}$	6.22$\times {10}^{-1}$	6.21$\times {10}^{-1}$
		±4.24$\times {10}^{-2}$	±3.96$\times {10}^{-2}$	±4.48$\times {10}^{-2}$	±2.04$\times {10}^{-2}$	±2.10$\times {10}^{-2}$	±2.19$\times {10}^{-2}$	±2.30$\times {10}^{-2}$	±1.60$\times {10}^{-2}$
15	Brain1	8.10$\times {10}^{-1}$	7.80$\times {10}^{-1}$	7.87$\times {10}^{-1}$	5.12$\times {10}^{-1}$	4.84$\times {10}^{-1}$	4.97$\times {10}^{-1}$	5.13$\times {10}^{-1}$	5.12$\times {10}^{-1}$
		±2.57$\times {10}^{-2}$	±4.48$\times {10}^{-2}$	±3.78$\times {10}^{-2}$	±5.71$\times {10}^{-3}$	±1.45$\times {10}^{-2}$	±2.42$\times {10}^{-3}$	±3.44$\times {10}^{-3}$	±4.38$\times {10}^{-3}$
16	Leukemia2	9.46$\times {10}^{-1}$	⋆ 9.27$\times {10}^{-1}$	⋆ 9.41$\times {10}^{-1}$	5.70$\times {10}^{-1}$	5.47$\times {10}^{-1}$	5.60$\times {10}^{-1}$	5.69$\times {10}^{-1}$	5.71$\times {10}^{-1}$
		±4.75$\times {10}^{-2}$	±5.28$\times {10}^{-2}$	±4.93$\times {10}^{-2}$	±2.09$\times {10}^{-2}$	±1.87$\times {10}^{-2}$	±1.82$\times {10}^{-2}$	±1.46$\times {10}^{-2}$	±1.83$\times {10}^{-2}$
17	CNS	7.12$\times {10}^{-1}$	⋆ 6.90$\times {10}^{-1}$	⋆ 6.95$\times {10}^{-1}$	3.97$\times {10}^{-1}$	3.88$\times {10}^{-1}$	3.85$\times {10}^{-1}$	4.10$\times {10}^{-1}$	4.02$\times {10}^{-1}$
		±6.10$\times {10}^{-2}$	±7.01$\times {10}^{-2}$	±7.90$\times {10}^{-2}$	±2.89$\times {10}^{-2}$	±3.18$\times {10}^{-2}$	±3.15$\times {10}^{-2}$	±3.47$\times {10}^{-2}$	±3.53$\times {10}^{-2}$
18	Arcene	8.70$\times {10}^{-1}$	⋆ 8.56$\times {10}^{-1}$	⋆ 8.65$\times {10}^{-1}$	3.84$\times {10}^{-1}$	3.56$\times {10}^{-1}$	3.77$\times {10}^{-1}$	3.84$\times {10}^{-1}$	3.84$\times {10}^{-1}$
		±2.37$\times {10}^{-2}$	±5.73$\times {10}^{-2}$	±2.21$\times {10}^{-2}$	±2.20$\times {10}^{-3}$	±1.35$\times {10}^{-3}$	±1.95$\times {10}^{-3}$	±1.44$\times {10}^{-3}$	±2.20$\times {10}^{-3}$
19	Brain2	7.33$\times {10}^{-1}$	6.99$\times {10}^{-1}$	⋆ 7.15$\times {10}^{-1}$	4.04$\times {10}^{-1}$	4.01$\times {10}^{-1}$	4.06$\times {10}^{-1}$	4.09$\times {10}^{-1}$	4.05$\times {10}^{-1}$
		±6.86$\times {10}^{-2}$	±6.26$\times {10}^{-2}$	±7.32$\times {10}^{-2}$	±2.46$\times {10}^{-2}$	±1.97$\times {10}^{-2}$	±2.73$\times {10}^{-2}$	±2.30$\times {10}^{-2}$	±2.53$\times {10}^{-2}$
20	Prostate	9.48$\times {10}^{-1}$	9.26$\times {10}^{-1}$	⋆ 9.42$\times {10}^{-1}$	4.81$\times {10}^{-1}$	4.75$\times {10}^{-1}$	4.76$\times {10}^{-1}$	4.80$\times {10}^{-1}$	4.77$\times {10}^{-1}$
		±3.05$\times {10}^{-2}$	±4.36$\times {10}^{-2}$	±2.72$\times {10}^{-2}$	±1.74$\times {10}^{-2}$	±1.36$\times {10}^{-2}$	±1.92$\times {10}^{-2}$	±1.68$\times {10}^{-2}$	±1.82$\times {10}^{-2}$

presents the HV performance statistics of all the comparison algorithms on each test dataset. It can be seen that DTEA performs the best on all cases, showing excellent convergence on relatively high-dimensional datasets (No. 12∼20) and promising diversity on relatively low-dimensional ones (No. 1∼11). It is indicated that DTEA adapts smoothly from low-dimensional to high-dimensional datasets, owing to its adaptive multi-tasking mechanism that dynamically reacts to real-time evolutionary demands. As a supplement,

Table 3. Mean IGD performances of each algorithm on each dataset, with best ones marked in gray, and insignificant differences prefixed by ⋆.

No.	Data	DTEA	SparseEA	DAEA	GFMMOEA	MOEAHD	ARMOEA	NSGAII	HypE
1	Wine	8.77$\times {10}^{-2}$	⋆ 8.97$\times {10}^{-2}$	⋆ 8.86$\times {10}^{-2}$	9.83$\times {10}^{-2}$	⋆ 8.79$\times {10}^{-2}$	⋆ 8.99$\times {10}^{-2}$	⋆ 9.04$\times {10}^{-2}$	⋆ 9.16$\times {10}^{-2}$
		±1.19$\times {10}^{-2}$	±1.22$\times {10}^{-2}$	±1.22$\times {10}^{-2}$	±1.35$\times {10}^{-2}$	±1.18$\times {10}^{-2}$	±1.29$\times {10}^{-2}$	±1.22$\times {10}^{-2}$	±1.25$\times {10}^{-2}$
2	Australian	1.91$\times {10}^{-2}$	⋆ 2.02$\times {10}^{-2}$	⋆ 1.91$\times {10}^{-2}$	3.71$\times {10}^{-2}$	⋆ 1.86$\times {10}^{-2}$	⋆ 2.13$\times {10}^{-2}$	⋆ 1.96$\times {10}^{-2}$	⋆ 2.12$\times {10}^{-2}$
		±5.52$\times {10}^{-3}$	±6.44$\times {10}^{-3}$	±5.50$\times {10}^{-3}$	±2.28$\times {10}^{-2}$	±5.85$\times {10}^{-3}$	±7.14$\times {10}^{-3}$	±7.13$\times {10}^{-3}$	±6.60$\times {10}^{-3}$
3	Climate	5.91$\times {10}^{-2}$	⋆ 6.09$\times {10}^{-2}$	⋆ 5.91$\times {10}^{-2}$	8.78$\times {10}^{-2}$	⋆ 6.55$\times {10}^{-2}$	⋆ 6.53$\times {10}^{-2}$	⋆ 6.49$\times {10}^{-2}$	⋆ 6.20$\times {10}^{-2}$
		±2.98$\times {10}^{-2}$	±3.18$\times {10}^{-2}$	±3.01$\times {10}^{-2}$	±3.80$\times {10}^{-2}$	±3.71$\times {10}^{-2}$	±3.71$\times {10}^{-2}$	±3.51$\times {10}^{-2}$	±3.16$\times {10}^{-2}$
4	German	4.05$\times {10}^{-2}$	⋆ 3.78$\times {10}^{-2}$	⋆ 3.87$\times {10}^{-2}$	6.35$\times {10}^{-2}$	⋆ 4.05$\times {10}^{-2}$	⋆ 3.76$\times {10}^{-2}$	⋆ 3.70$\times {10}^{-2}$	⋆ 4.35$\times {10}^{-2}$
		±1.75$\times {10}^{-2}$	±1.55$\times {10}^{-2}$	±1.57$\times {10}^{-2}$	±2.63$\times {10}^{-2}$	±1.45$\times {10}^{-2}$	±9.16$\times {10}^{-3}$	±1.16$\times {10}^{-2}$	±1.52$\times {10}^{-2}$
5	Breast	1.96$\times {10}^{-2}$	⋆ 1.92$\times {10}^{-2}$	⋆ 1.96$\times {10}^{-2}$	2.66$\times {10}^{-2}$	⋆ 1.95$\times {10}^{-2}$	⋆ 2.15$\times {10}^{-2}$	⋆ 2.01$\times {10}^{-2}$	2.44$\times {10}^{-2}$
		±4.94$\times {10}^{-3}$	±4.14$\times {10}^{-3}$	±3.44$\times {10}^{-3}$	±1.31$\times {10}^{-2}$	±5.37$\times {10}^{-3}$	±7.50$\times {10}^{-3}$	±6.52$\times {10}^{-3}$	±6.85$\times {10}^{-3}$
6	Ionosphere	3.92$\times {10}^{-2}$	⋆ 4.18$\times {10}^{-2}$	⋆ 4.04$\times {10}^{-2}$	5.12$\times {10}^{-2}$	⋆ 3.97$\times {10}^{-2}$	⋆ 4.09$\times {10}^{-2}$	⋆ 4.30$\times {10}^{-2}$	⋆ 3.97$\times {10}^{-2}$
		±5.93$\times {10}^{-3}$	±8.96$\times {10}^{-3}$	±5.06$\times {10}^{-3}$	±1.08$\times {10}^{-2}$	±5.91$\times {10}^{-3}$	±9.80$\times {10}^{-3}$	±8.12$\times {10}^{-3}$	±7.55$\times {10}^{-3}$
7	Lung	1.03$\times {10}^{-1}$	⋆ 1.10$\times {10}^{-1}$	⋆ 1.06$\times {10}^{-1}$	1.48$\times {10}^{-1}$	1.32$\times {10}^{-1}$	1.27$\times {10}^{-1}$	1.29$\times {10}^{-1}$	⋆ 1.14$\times {10}^{-1}$
		±3.25$\times {10}^{-2}$	±3.78$\times {10}^{-2}$	±3.21$\times {10}^{-2}$	±5.62$\times {10}^{-2}$	±3.70$\times {10}^{-2}$	±4.53$\times {10}^{-2}$	±4.52$\times {10}^{-2}$	±4.05$\times {10}^{-2}$
8	Sonar	6.20$\times {10}^{-2}$	⋆ 6.66$\times {10}^{-2}$	⋆ 6.93$\times {10}^{-2}$	8.43$\times {10}^{-2}$	⋆ 7.12$\times {10}^{-2}$	7.73$\times {10}^{-2}$	⋆ 6.82$\times {10}^{-2}$	7.51$\times {10}^{-2}$
		±1.70$\times {10}^{-2}$	±1.93$\times {10}^{-2}$	±2.15$\times {10}^{-2}$	±2.69$\times {10}^{-2}$	±2.11$\times {10}^{-2}$	±2.20$\times {10}^{-2}$	±2.32$\times {10}^{-2}$	±2.92$\times {10}^{-2}$
9	Hill_Valley	2.71$\times {10}^{-2}$	⋆ 2.92$\times {10}^{-2}$	⋆ 2.84$\times {10}^{-2}$	5.82$\times {10}^{-2}$	⋆ 2.91$\times {10}^{-2}$	5.00$\times {10}^{-2}$	3.99$\times {10}^{-2}$	⋆ 3.15$\times {10}^{-2}$
		±6.07$\times {10}^{-3}$	±6.27$\times {10}^{-3}$	±9.56$\times {10}^{-3}$	±2.33$\times {10}^{-2}$	±9.17$\times {10}^{-3}$	±2.46$\times {10}^{-2}$	±2.13$\times {10}^{-2}$	±1.35$\times {10}^{-2}$
10	Semeion	2.31$\times {10}^{-2}$	2.90$\times {10}^{-2}$	2.71$\times {10}^{-2}$	1.88$\times {10}^{-1}$	1.11$\times {10}^{-1}$	1.72$\times {10}^{-1}$	1.63$\times {10}^{-1}$	1.80$\times {10}^{-1}$
		±3.29$\times {10}^{-3}$	±2.99$\times {10}^{-3}$	±7.95$\times {10}^{-3}$	±1.86$\times {10}^{-2}$	±1.77$\times {10}^{-2}$	±1.92$\times {10}^{-2}$	±1.61$\times {10}^{-2}$	±1.02$\times {10}^{-2}$
11	Madelon	1.90$\times {10}^{-2}$	⋆ 2.03$\times {10}^{-2}$	⋆ 1.73$\times {10}^{-2}$	2.32$\times {10}^{-1}$	1.76$\times {10}^{-1}$	2.23$\times {10}^{-1}$	2.11$\times {10}^{-1}$	1.86$\times {10}^{-1}$
		±4.63$\times {10}^{-3}$	±4.58$\times {10}^{-3}$	±4.95$\times {10}^{-3}$	±4.09$\times {10}^{-2}$	±1.90$\times {10}^{-2}$	±3.92$\times {10}^{-2}$	±3.02$\times {10}^{-2}$	±2.72$\times {10}^{-2}$
12	SRBCT	4.12$\times {10}^{-2}$	⋆ 5.22$\times {10}^{-2}$	5.31$\times {10}^{-2}$	5.69$\times {10}^{-1}$	6.33$\times {10}^{-1}$	5.91$\times {10}^{-1}$	5.60$\times {10}^{-1}$	5.69$\times {10}^{-1}$
		±1.67$\times {10}^{-2}$	±2.32$\times {10}^{-2}$	±2.21$\times {10}^{-2}$	±2.74$\times {10}^{-3}$	±2.87$\times {10}^{-3}$	±3.30$\times {10}^{-3}$	±5.23$\times {10}^{-2}$	±2.74$\times {10}^{-3}$
13	Leukemia1	4.93$\times {10}^{-2}$	⋆ 5.08$\times {10}^{-2}$	⋆ 5.15$\times {10}^{-2}$	3.93$\times {10}^{-1}$	4.08$\times {10}^{-1}$	4.11$\times {10}^{-1}$	3.89$\times {10}^{-1}$	3.97$\times {10}^{-1}$
		±2.32$\times {10}^{-2}$	±1.86$\times {10}^{-2}$	±1.88$\times {10}^{-2}$	±1.12$\times {10}^{-2}$	±1.08$\times {10}^{-2}$	±1.19$\times {10}^{-2}$	±7.76$\times {10}^{-3}$	±1.08$\times {10}^{-2}$
14	DLBCL	6.17$\times {10}^{-2}$	9.13$\times {10}^{-2}$	⋆ 8.34$\times {10}^{-2}$	3.75$\times {10}^{-1}$	3.96$\times {10}^{-1}$	3.96$\times {10}^{-1}$	3.76$\times {10}^{-1}$	3.81$\times {10}^{-1}$
		±4.34$\times {10}^{-2}$	±4.36$\times {10}^{-2}$	±4.78$\times {10}^{-2}$	±7.58$\times {10}^{-3}$	±1.13$\times {10}^{-2}$	±6.63$\times {10}^{-3}$	±7.15$\times {10}^{-3}$	±4.45$\times {10}^{-3}$
15	Brain1	3.78$\times {10}^{-2}$	6.61$\times {10}^{-2}$	5.58$\times {10}^{-2}$	3.72$\times {10}^{-1}$	4.11$\times {10}^{-1}$	3.94$\times {10}^{-1}$	3.71$\times {10}^{-1}$	3.73$\times {10}^{-1}$
		±1.49$\times {10}^{-2}$	±3.84$\times {10}^{-2}$	±3.39$\times {10}^{-2}$	±8.02$\times {10}^{-3}$	±2.04$\times {10}^{-2}$	±3.41$\times {10}^{-3}$	±4.83$\times {10}^{-3}$	±6.15$\times {10}^{-3}$
16	Leukemia2	5.91$\times {10}^{-2}$	⋆ 8.03$\times {10}^{-2}$	⋆ 6.52$\times {10}^{-2}$	4.07$\times {10}^{-1}$	4.33$\times {10}^{-1}$	4.22$\times {10}^{-1}$	4.07$\times {10}^{-1}$	4.07$\times {10}^{-1}$
		±5.22$\times {10}^{-2}$	±5.81$\times {10}^{-2}$	±5.43$\times {10}^{-2}$	±1.43$\times {10}^{-2}$	±1.49$\times {10}^{-2}$	±1.03$\times {10}^{-2}$	±1.03$\times {10}^{-2}$	±1.06$\times {10}^{-2}$
17	CNS	2.06$\times {10}^{-1}$	⋆ 2.30$\times {10}^{-1}$	⋆ 2.24$\times {10}^{-1}$	5.00$\times {10}^{-1}$	5.12$\times {10}^{-1}$	5.17$\times {10}^{-1}$	4.87$\times {10}^{-1}$	4.97$\times {10}^{-1}$
		±6.71$\times {10}^{-2}$	±7.68$\times {10}^{-2}$	±8.69$\times {10}^{-2}$	±3.14$\times {10}^{-2}$	±3.43$\times {10}^{-2}$	±3.45$\times {10}^{-2}$	±3.64$\times {10}^{-2}$	±3.51$\times {10}^{-2}$
18	Arcene	4.56$\times {10}^{-2}$	⋆ 5.98$\times {10}^{-2}$	⋆ 4.80$\times {10}^{-2}$	5.19$\times {10}^{-1}$	5.58$\times {10}^{-1}$	5.29$\times {10}^{-1}$	5.19$\times {10}^{-1}$	5.19$\times {10}^{-1}$
		±1.77$\times {10}^{-2}$	±5.48$\times {10}^{-2}$	±1.73$\times {10}^{-2}$	±3.10$\times {10}^{-3}$	±1.99$\times {10}^{-3}$	±2.79$\times {10}^{-3}$	±2.04$\times {10}^{-3}$	±3.10$\times {10}^{-3}$
19	Brain2	1.04$\times {10}^{-1}$	1.38$\times {10}^{-1}$	⋆ 1.22$\times {10}^{-1}$	4.59$\times {10}^{-1}$	4.67$\times {10}^{-1}$	4.65$\times {10}^{-1}$	4.55$\times {10}^{-1}$	4.61$\times {10}^{-1}$
		±6.29$\times {10}^{-2}$	±5.79$\times {10}^{-2}$	±6.96$\times {10}^{-2}$	±1.80$\times {10}^{-2}$	±1.39$\times {10}^{-2}$	±1.88$\times {10}^{-2}$	±1.67$\times {10}^{-2}$	±1.67$\times {10}^{-2}$
20	Prostate	3.88$\times {10}^{-2}$	⋆ 5.56$\times {10}^{-2}$	⋆ 3.77$\times {10}^{-2}$	4.69$\times {10}^{-1}$	4.79$\times {10}^{-1}$	4.77$\times {10}^{-1}$	4.72$\times {10}^{-1}$	4.74$\times {10}^{-1}$
		±2.34$\times {10}^{-2}$	±3.98$\times {10}^{-2}$	±2.33$\times {10}^{-2}$	±1.47$\times {10}^{-2}$	±1.11$\times {10}^{-2}$	±1.59$\times {10}^{-2}$	±1.42$\times {10}^{-2}$	±1.55$\times {10}^{-2}$

shows the statistic IGD performances, while DTEA performs the best on most of the test instances, but loses insignificantly on five of them. However, overall, the proposed DTEA is still the best among all the comparison algorithms, which is also testified by the Friedman test results shown in

Table 4. Mean ranks calculated by Friedman’s test on all datasets.

Metric	DTEA	SparseEA	DAEA	GFMMOEA	MOEAHD	ARMOEA	NSGAII	HypE
HV	1st	3rd	2nd	8th	6th	7th	4th	5th
IGD	1st	3rd	2nd	8th	6th	7th	4th	5th
MCE	1st	3rd	2nd	8th	6th	7th	5th	4th

. Table 4 gives the average Friedman test rankings for all the algorithms in terms of the HV, IGD and MCE metrics, respectively, while the MCE performance will be discussed later in Section 5.3.

Furthermore, although DTEA performs better than all the other algorithms in terms of the HV metric on every dataset, as observed from Table 2, it still has some slight setbacks in terms of the IGD metric on five out of 20 datasets, as observed from Table 3. In Table 3, DTEA loses insignificantly on the Australian, German, Breast, Madelon and Prostate datasets; the first four are relatively low-dimensional datasets that cannot fully utilize the large-scale search capability of the proposed algorithm. Moreover, on the Prostate dataset, DTEA loses only insignificantly to DAEA, which is also specifically designed for tackling high-dimensional feature selection, and performs much better than all the other algorithms. It should also be noted that IGD is adopted as a supplemental performance indicator in this paper since it relies highly on the sampled reference points and may not be as accurate as the major performance indicator HV shown in Table 2 whose reference point is fixed as (1, 1) in the objective space.

Finally, the mean ranks calculated using Friedman’s test shown in Table 4 also verify the overall performance advantages of the proposed DTEA over the other algorithms on all datasets in terms of all three metrics (MCE will be discussed later). It can also be observed that, although still worse than DTEA, the comparison algorithm DAEA generally performs second, while SparseEA comes next; the latter are two recently published methods specifically designed for tackling bi-objective feature selection.

5.2. Solution Distributions

The nondominated solution distributions of the test data for each algorithm are illustrated in Figure 5, with the corresponding median HV performance among all 30 independent runs for the sake of fairness. Six datasets, i.e., Breast, Sonar, Leukemia1, Leukemia2, CNS and Brain2 are chosen to represent the relatively lower and higher dimensionality of features. It can be seen that the proposed algorithm DTEA generally performs the best among all eight algorithms, with quite promising population diversity and much better classification convergence.

In detail, in two low-dimensional cases (i.e., the Breast and Sonar datasets), the advantage of DTEA is not so significant, but it can still be observed that DTEA obtains the best converged solution in the direction of the $f_2$ objective (i.e., the classification error rate) and also reaches the best $f_1$ objective value. By contrast, the advantage of DTEA is more significant in the other high-dimensional cases (i.e., the Leukemia1, Leukemia2, CNS and Brain2 datasets), as can be observed by DTEA maintaining more diverse population distributions with more nondominated solutions obtained, along with the best converged solution in the $f_2$ objective direction. This is because the search space in high-dimensional cases becomes much sparser than that in low-dimensional cases, making the feasible solutions much rarer and more difficult to be found. In this situation, the superior search ability of DTEA, enhanced by its dynamic multi-tasking mechanism, helps to better explore the potential feasible solutions, thereby finding more nondominated ones.

5.3. Classification Performances

Table 5. Mean MCE and SFN performances of each algorithm on each dataset, with best ones marked in gray, and insignificant differences prefixed by ⋆.

No.	Data	DTEA	SparseEA	DAEA	GFMMOEA	MOEAHD	ARMOEA	NSGAII	HypE
1	Wine	1.45$\times {10}^{-2}$	⋆ 2.01$\times {10}^{-2}$	⋆ 1.64$\times {10}^{-2}$	2.45$\times {10}^{-2}$	⋆ 1.57$\times {10}^{-2}$	⋆ 2.26$\times {10}^{-2}$	⋆ 1.95$\times {10}^{-2}$	⋆ 1.95$\times {10}^{-2}$
		SFN = 5	SFN = 6	SFN = 6	SFN = 6	SFN = 5	SFN = 5	SFN = 6	SFN = 6
2	Australian	1.46$\times {10}^{-1}$	⋆  1.50$\times {10}^{-1}$	⋆ 1.46$\times {10}^{-1}$	⋆ 1.51$\times {10}^{-1}$	⋆ 1.47$\times {10}^{-1}$	⋆ 1.47$\times {10}^{-1}$	⋆ 1.46$\times {10}^{-1}$	⋆ 1.47$\times {10}^{-1}$
		SFN = 4	SFN = 4	SFN = 4	SFN = 5	SFN = 4	SFN = 4	SFN = 4	SFN = 4
3	Climate	6.42$\times {10}^{-2}$	⋆ 6.56$\times {10}^{-2}$	⋆ 6.44$\times {10}^{-2}$	7.18$\times {10}^{-2}$	⋆ 6.56$\times {10}^{-2}$	⋆ 6.56$\times {10}^{-2}$	⋆ 6.54$\times {10}^{-2}$	⋆ 6.46$\times {10}^{-2}$
		SFN = 5	SFN = 5	SFN = 5	SFN = 5	SFN = 5	SFN = 5	SFN = 5	SFN = 5
4	German	2.64$\times {10}^{-1}$	⋆ 2.64$\times {10}^{-1}$	⋆ 2.65$\times {10}^{-1}$	2.75$\times {10}^{-1}$	⋆ 2.64$\times {10}^{-1}$	⋆ 2.66$\times {10}^{-1}$	⋆ 2.64$\times {10}^{-1}$	⋆ 2.65$\times {10}^{-1}$
		SFN = 6	SFN = 6	SFN = 6	SFN = 6	SFN = 6	SFN = 6	SFN = 6	SFN = 6
5	Breast	3.04$\times {10}^{-2}$	⋆ 3.26$\times {10}^{-2}$	⋆ 3.18$\times {10}^{-2}$	3.33$\times {10}^{-2}$	3.33$\times {10}^{-2}$	⋆ 3.16$\times {10}^{-2}$	⋆ 3.16$\times {10}^{-2}$	⋆ 3.08$\times {10}^{-2}$
		SFN = 6	SFN = 5	SFN = 6	SFN = 5	SFN = 5	SFN = 5	SFN = 5	SFN = 5
6	Ionosphere	9.43$\times {10}^{-2}$	⋆ 9.87$\times {10}^{-2}$	⋆ 9.65$\times {10}^{-2}$	1.08$\times {10}^{-1}$	⋆ 9.43$\times {10}^{-2}$	⋆ 9.62$\times {10}^{-2}$	⋆ 9.84$\times {10}^{-2}$	⋆ 9.65$\times {10}^{-2}$
		SFN = 3	SFN = 3	SFN = 3	SFN = 4	SFN = 3	SFN = 3	SFN = 3	SFN = 3
7	Lung	2.63$\times {10}^{-1}$	⋆ 2.73$\times {10}^{-1}$	⋆ 2.70$\times {10}^{-1}$	3.33$\times {10}^{-1}$	3.20$\times {10}^{-1}$	⋆ 2.90$\times {10}^{-1}$	⋆ 3.03$\times {10}^{-1}$	⋆ 2.93$\times {10}^{-1}$
		SFN = 6	SFN = 6	SFN = 6	SFN = 6	SFN = 4	SFN = 7	SFN = 5	SFN = 5
8	Sonar	1.85$\times {10}^{-1}$	⋆ 1.92$\times {10}^{-1}$	⋆ 1.96$\times {10}^{-1}$	2.12$\times {10}^{-1}$	2.04$\times {10}^{-1}$	2.05$\times {10}^{-1}$	⋆ 1.98$\times {10}^{-1}$	⋆ 1.98$\times {10}^{-1}$
		SFN = 8	SFN = 8	SFN = 7	SFN = 6	SFN = 5	SFN = 6	SFN = 6	SFN = 8
9	Hill_Valley	3.97$\times {10}^{-1}$	⋆ 4.01$\times {10}^{-1}$	⋆ 4.00$\times {10}^{-1}$	4.22$\times {10}^{-1}$	4.04$\times {10}^{-1}$	4.15$\times {10}^{-1}$	4.09$\times {10}^{-1}$	4.04$\times {10}^{-1}$
		SFN = 5	SFN = 5	SFN = 5	SFN = 10	SFN = 4	SFN = 8	SFN = 6	SFN = 4
10	Semeion	1.35$\times {10}^{-1}$	⋆ 1.40$\times {10}^{-1}$	⋆ 1.31$\times {10}^{-1}$	⋆ 1.38$\times {10}^{-1}$	⋆ 1.37$\times {10}^{-1}$	⋆ 1.37$\times {10}^{-1}$	⋆ 1.37$\times {10}^{-1}$	⋆ 1.38$\times {10}^{-1}$
		SFN = 80	SFN = 97	SFN = 82	SFN = 89	SFN = 87	SFN = 84	SFN = 88	SFN = 79
11	Madelon	9.73$\times {10}^{-2}$	9.95$\times {10}^{-2}$	⋆ 9.77$\times {10}^{-2}$	3.09$\times {10}^{-1}$	2.74$\times {10}^{-1}$	2.99$\times {10}^{-1}$	2.92$\times {10}^{-1}$	2.75$\times {10}^{-1}$
		SFN = 7	SFN = 8	SFN = 7	SFN = 100	SFN = 66	SFN = 92	SFN = 89	SFN = 71
12	SRBCT	4.00$\times {10}^{-2}$	⋆ 6.13$\times {10}^{-2}$	7.20$\times {10}^{-2}$	6.40$\times {10}^{-1}$	6.40$\times {10}^{-1}$	6.40$\times {10}^{-1}$	6.24$\times {10}^{-1}$	6.40$\times {10}^{-1}$
		SFN = 5	SFN = 4	SFN = 4	SFN = 579	SFN = 854	SFN = 684	SFN = 582	SFN = 579
13	Leukemia1	5.00$\times {10}^{-2}$	⋆ 5.15$\times {10}^{-2}$	⋆ 5.76$\times {10}^{-2}$	1.71$\times {10}^{-1}$	1.67$\times {10}^{-1}$	1.70$\times {10}^{-1}$	1.64$\times {10}^{-1}$	1.76$\times {10}^{-1}$
		SFN = 2	SFN = 3	SFN = 3	SFN = 1980	SFN = 2078	SFN = 2084	SFN = 1976	SFN = 2000
14	DLBCL	8.12$\times {10}^{-2}$	1.13$\times {10}^{-1}$	⋆ 1.04$\times {10}^{-1}$	⋆ 6.09$\times {10}^{-2}$	⋆ 6.81$\times {10}^{-2}$	⋆ 6.23$\times {10}^{-2}$	⋆ 6.81$\times {10}^{-2}$	5.94$\times {10}^{-2}$
		SFN = 3	SFN = 3	SFN = 2	SFN = 2031	SFN = 2142	SFN = 2144	SFN = 2030	SFN = 2064
15	Brain1	2.09$\times {10}^{-1}$	2.42$\times {10}^{-1}$	2.35$\times {10}^{-1}$	2.59$\times {10}^{-1}$	2.59$\times {10}^{-1}$	2.59$\times {10}^{-1}$	2.59$\times {10}^{-1}$	2.59$\times {10}^{-1}$
		SFN = 4	SFN = 3	SFN = 5	SFN = 2147	SFN = 2387	SFN = 2282	SFN = 2141	SFN = 2154
16	Leukemia2	5.91$\times {10}^{-2}$	⋆ 8.03$\times {10}^{-2}$	⋆ 6.52$\times {10}^{-2}$	1.42$\times {10}^{-1}$	1.42$\times {10}^{-1}$	1.32$\times {10}^{-1}$	1.42$\times {10}^{-1}$	1.36$\times {10}^{-1}$
		SFN = 2	SFN = 2	SFN = 2	SFN = 2688	SFN = 2889	SFN = 2830	SFN = 2692	SFN = 2707
17	CNS	3.17$\times {10}^{-1}$	⋆ 3.41$\times {10}^{-1}$	⋆ 3.35$\times {10}^{-1}$	4.28$\times {10}^{-1}$	4.31$\times {10}^{-1}$	4.31$\times {10}^{-1}$	4.07$\times {10}^{-1}$	4.15$\times {10}^{-1}$
		SFN = 3	SFN = 5	SFN = 2	SFN = 2750	SFN = 2837	SFN = 2877	SFN = 2739	SFN = 2795
18	Arcene	1.43$\times {10}^{-1}$	⋆ 1.57$\times {10}^{-1}$	⋆ 1.49$\times {10}^{-1}$	4.33$\times {10}^{-1}$	4.33$\times {10}^{-1}$	4.33$\times {10}^{-1}$	4.33$\times {10}^{-1}$	4.33$\times {10}^{-1}$
		SFN = 7	SFN = 27	SFN = 7	SFN = 4037	SFN = 4533	SFN = 4159	SFN = 4036	SFN = 4037
19	Brain2	2.93$\times {10}^{-1}$	⋆ 3.31$\times {10}^{-1}$	⋆ 3.13$\times {10}^{-1}$	3.89$\times {10}^{-1}$	3.82$\times {10}^{-1}$	3.73$\times {10}^{-1}$	3.80$\times {10}^{-1}$	3.82$\times {10}^{-1}$
		SFN = 3	SFN = 8	SFN = 3	SFN = 4271	SFN = 4407	SFN = 4402	SFN = 4275	SFN = 4319
20	Prostate	5.70$\times {10}^{-2}$	8.06$\times {10}^{-2}$	⋆ 6.34$\times {10}^{-2}$	2.53$\times {10}^{-1}$	2.48$\times {10}^{-1}$	2.47$\times {10}^{-1}$	2.55$\times {10}^{-1}$	2.56$\times {10}^{-1}$
		SFN = 3	SFN = 13	SFN = 4	SFN = 4338	SFN = 4475	SFN = 4457	SFN = 4353	SFN = 4377

exhibits the classification performances of all eight comparison algorithms on each test dataset. For the readers’ convenience, the mean minimum classification error (MCE) and the corresponding selected feature number (SFN) values are shown up and down in two rows together. It can be seen that DTEA performs the best on most of the datasets with the smallest MCE values and moderately small SFN values, while only two slight losses occur on the No. 10 and No. 14 datasets. To be more specific, the No. 10 dataset (Semeion) is a relatively low-dimensional dataset with 256 features, where DTEA only loses insignificantly to DAEA as the second place. By contrast, the No. 14 dataset (DLBCL) is a high-dimensional instance with 5469 features, where the three large-scale problem specialized algorithms (i.e., DTEA, SparseEA and DAEA) all lose to the other five algorithms (i.e., GFMMOEA, MOEAHD, ARMOEA, NSGAII and HypE), although the corresponding HV and IGD performances of DTEA are obviously the best. Overall, the winning rate of DTEA against all the other comparison algorithms is up to $90\%$ in terms of MCE (still ranking first in Table 4), not to mention the much smaller SFN values in high-dimensional cases that can greatly reduce the classification computational cost.

Moreover, by comparing the SFN results, it is found that all the algorithms can find moderately small sizes of feature subsets on relatively low-dimensional datasets; however, as the dimension rises, the SFN values from DTEA, SparseEA and DAEA become much smaller than those from GFMMOEA, MOEAHD, ARMOEA, NSGAII and HypE. This provides an explanation for the MCE performances presented on the No. 14 dataset DLBCL, where DTEA, SparseEA and DAEA might lose some useful features when accelerating the population convergence, which are difficult to recover in the later evolutionary search.

Nevertheless, DTEA still performs better than SparseEA and DAEA on DLBCL, and not significantly different from GFMMOEA, MOEAHD, ARMOEA and NSGAII, proving its capability of alleviating the overfitting or early maturity to some extent.

5.4. Component Contributions

The component contributions of DTEA on the backward and ROI search tasks are analyzed by comparing DTEA and three of its variants (i.e., DTEA_Back, DTEA_ROI and DTEA_Main, as previously introduced in Section 4.1). Here, DTEA_Back, DTEA_ROI and DTEA_Main only remain the backward and main tasks, the ROI and main tasks, and the single main task, respectively. It should be noted that DTEA_Main still adopts the duplication removal mechanism (Line 4 in Algorithm 3) and uses the same modified reproduction method as DTEA, DTEA_Back and DTEA_ROI. Thus, the single-tasking DTEA_Main is understood to perform better in feature selection than the basic framework of MO-MFEA [37]. Overall, as shown in

Table 6. Mean HV performances of each algorithm on each dataset, with best ones marked in gray, and those performances better than DTEA_Main prefixed by $\Delta$.

No.	Data	DTEA	DTEA_Back	DTEA_ROI	DTEA_Main
1	Wine	$\Delta$ 8.7802$\times {10}^{-1}$	8.7756$\times {10}^{-1}$	$\Delta$ 8.7802$\times {10}^{-1}$	8.7756$\times {10}^{-1}$
		±1.02$\times {10}^{-2}$	±1.03$\times {10}^{-2}$	±1.02$\times {10}^{-2}$	±1.03$\times {10}^{-2}$
2	Australian	$\Delta$ 7.9918$\times {10}^{-1}$	7.9894$\times {10}^{-1}$	$\Delta$ 7.9918$\times {10}^{-1}$	7.9894$\times {10}^{-1}$
		±5.48$\times {10}^{-3}$	±5.19$\times {10}^{-3}$	±5.48$\times {10}^{-3}$	±5.19$\times {10}^{-3}$
3	Climate	8.9031$\times {10}^{-1}$	8.9031$\times {10}^{-1}$	8.9031$\times {10}^{-1}$	8.9031$\times {10}^{-1}$
		±6.44$\times {10}^{-3}$	±6.44$\times {10}^{-3}$	±6.44$\times {10}^{-3}$	±6.44$\times {10}^{-3}$
4	German	$\Delta$ 7.2807$\times {10}^{-1}$	7.2687$\times {10}^{-1}$	$\Delta$ 7.2807$\times {10}^{-1}$	7.2687$\times {10}^{-1}$
		±7.83$\times {10}^{-3}$	±6.73$\times {10}^{-3}$	±7.83$\times {10}^{-3}$	±6.73$\times {10}^{-3}$
5	Breast	$\Delta$ 9.3969$\times {10}^{-1}$	9.3938$\times {10}^{-1}$	$\Delta$ 9.3969$\times {10}^{-1}$	9.3938$\times {10}^{-1}$
		±4.40$\times {10}^{-3}$	±4.84$\times {10}^{-3}$	±4.40$\times {10}^{-3}$	±4.84$\times {10}^{-3}$
6	Ionosphere	$\Delta$ 8.8640$\times {10}^{-1}$	8.8614$\times {10}^{-1}$	$\Delta$ 8.8640$\times {10}^{-1}$	8.8614$\times {10}^{-1}$
		±8.75$\times {10}^{-3}$	±8.62$\times {10}^{-3}$	±8.75$\times {10}^{-3}$	±8.62$\times {10}^{-3}$
7	Lung	7.3808$\times {10}^{-1}$	7.4643$\times {10}^{-1}$	7.3808$\times {10}^{-1}$	7.4643$\times {10}^{-1}$
		±6.23$\times {10}^{-2}$	±6.37$\times {10}^{-2}$	±6.23$\times {10}^{-2}$	±6.37$\times {10}^{-2}$
8	Sonar	$\Delta$ 8.1310$\times {10}^{-1}$	8.0758$\times {10}^{-1}$	$\Delta$ 8.1310$\times {10}^{-1}$	8.0758$\times {10}^{-1}$
		±1.96$\times {10}^{-2}$	±2.05$\times {10}^{-2}$	±1.96$\times {10}^{-2}$	±2.05$\times {10}^{-2}$
9	Hill_Valley	6.3204$\times {10}^{-1}$	6.3196$\times {10}^{-1}$	$\Delta$ 6.3269$\times {10}^{-1}$	6.3261$\times {10}^{-1}$
		±7.98$\times {10}^{-3}$	±1.14$\times {10}^{-2}$	±9.52$\times {10}^{-3}$	±1.25$\times {10}^{-2}$
10	Semeion	$\Delta$ 8.4357$\times {10}^{-1}$	$\Delta$ 8.4357$\times {10}^{-1}$	7.9829$\times {10}^{-1}$	7.9829$\times {10}^{-1}$
		±6.83$\times {10}^{-3}$	±6.83$\times {10}^{-3}$	±1.56$\times {10}^{-2}$	±1.56$\times {10}^{-2}$
11	Madelon	$\Delta$ 9.0843$\times {10}^{-1}$	$\Delta$ 9.0835$\times {10}^{-1}$	9.0629$\times {10}^{-1}$	9.0632$\times {10}^{-1}$
		±3.47$\times {10}^{-3}$	±2.40$\times {10}^{-3}$	±3.82$\times {10}^{-3}$	±3.80$\times {10}^{-3}$
12	SRBCT	$\Delta$ 9.6300$\times {10}^{-1}$	$\Delta$ 9.5694$\times {10}^{-1}$	5.0049$\times {10}^{-1}$	5.0049$\times {10}^{-1}$
		±3.57$\times {10}^{-2}$	±4.27$\times {10}^{-2}$	±1.64$\times {10}^{-1}$	±1.64$\times {10}^{-1}$
13	Leukemia1	$\Delta$ 9.5436$\times {10}^{-1}$	$\Delta$ 9.5848$\times {10}^{-1}$	6.5524$\times {10}^{-1}$	6.5524$\times {10}^{-1}$
		±3.49$\times {10}^{-2}$	±2.66$\times {10}^{-2}$	±3.43$\times {10}^{-2}$	±3.43$\times {10}^{-2}$
14	DLBCL	$\Delta$ 9.2604$\times {10}^{-1}$	$\Delta$ 9.1945$\times {10}^{-1}$	7.3657$\times {10}^{-1}$	7.3657$\times {10}^{-1}$
		±4.24$\times {10}^{-2}$	±4.81$\times {10}^{-2}$	±3.28$\times {10}^{-2}$	±3.28$\times {10}^{-2}$
15	Brain1	$\Delta$ 8.1017$\times {10}^{-1}$	$\Delta$ 8.1017$\times {10}^{-1}$	5.9503$\times {10}^{-1}$	5.9503$\times {10}^{-1}$
		±2.57$\times {10}^{-2}$	±3.90$\times {10}^{-2}$	±1.16$\times {10}^{-2}$	±1.16$\times {10}^{-2}$
16	Leukemia2	$\Delta$ 9.4614$\times {10}^{-1}$	$\Delta$ 9.4201$\times {10}^{-1}$	6.6568$\times {10}^{-1}$	6.6568$\times {10}^{-1}$
		±4.75$\times {10}^{-2}$	±6.20$\times {10}^{-2}$	±1.98$\times {10}^{-2}$	±1.98$\times {10}^{-2}$
17	CNS	$\Delta$ 7.1200$\times {10}^{-1}$	$\Delta$ 7.0022$\times {10}^{-1}$	4.6720$\times {10}^{-1}$	4.6720$\times {10}^{-1}$
		±6.10$\times {10}^{-2}$	±7.25$\times {10}^{-2}$	±5.10$\times {10}^{-2}$	±5.10$\times {10}^{-2}$
18	Arcene	$\Delta$ 8.6957$\times {10}^{-1}$	$\Delta$ 8.6554$\times {10}^{-1}$	4.4171$\times {10}^{-1}$	4.4171$\times {10}^{-1}$
		±2.37$\times {10}^{-2}$	±2.21$\times {10}^{-2}$	±4.14$\times {10}^{-3}$	±4.14$\times {10}^{-3}$
19	Brain2	$\Delta$ 7.3324$\times {10}^{-1}$	$\Delta$ 7.2920$\times {10}^{-1}$	4.6463$\times {10}^{-1}$	4.6463$\times {10}^{-1}$
		±6.86$\times {10}^{-2}$	±6.11$\times {10}^{-2}$	±3.37$\times {10}^{-2}$	±3.37$\times {10}^{-2}$
20	Prostate	$\Delta$ 9.4809$\times {10}^{-1}$	$\Delta$ 9.4418$\times {10}^{-1}$	5.5444$\times {10}^{-1}$	5.5444$\times {10}^{-1}$
		±3.05$\times {10}^{-2}$	±2.59$\times {10}^{-2}$	±2.27$\times {10}^{-2}$	±2.27$\times {10}^{-2}$

, DTEA performs the best on most of the datasets against its three variants, with better performance on 17 out of all the 20 datasets, demonstrating the advantage of combining the two essential components (i.e., the backward and ROI search tasks) together.

Moreover, by observing the performance comparisons between DTEA_Back, DTEA_ROI and DTEA_Main, the contribution and effectiveness of the backward and ROI task components can be verified. Specifically, DTEA_Back performs the same with DTEA_Main on relatively low-dimensional datasets (No. 1∼8) and significantly better on many others (No. 10∼20), but loses a little on Hill_Valley, which has 100 features and stands between low and high dimensions. This actually proves the necessity and validity of our dynamic tasking mechanism of judging whether or not to add the backward task, because not all circumstances are suitable for convergence acceleration and the backward task is normally triggered in relatively high-dimensional cases.

By contrast, DTEA_ROI performs the same with DTEA_Main on relatively high-dimensional datasets (No. 10∼20), but better on all the others except for being the same with DTEA_Main on No. 3 and losing insignificantly on No. 7. This is because the No. 3 Climate dataset is too simple for the ROI task component to make any contribution, and the No. 7 Lung dataset has too few classification samples. Like the dynamic tasking mechanism of the backward task, the ROI task is not triggered unless the population stops convergence for at least 10 generations. Thus, without the acceleration help of the backward task, DTEA_ROI cannot reach the triggering criterion in the relatively high-dimensional cases.

In general, based on the above analyses, it can be concluded that either the backward or ROI task component can be adaptively judged to be added or not, and mostly produces benefits if triggered. Moreover, the combination of the two components, together constituting the dynamic multi-tasking system in DTEA, works efficiently and cooperatively, whether on low- or high-dimensional datasets.

6. Conclusions

In this paper, a dynamic multi-tasking evolutionary algorithm (DTEA) is proposed for solving the bi-objective feature selection problem, especially in high-dimensional cases. The dynamic multi-tasking system consists of judging, adding, merging and updating either the backward search task or the ROI one. The former is used to respond to the convergence evolutionary requirement, while the latter responds to the diversity need. To test its effectiveness, a total of 20 classification datasets, including relatively low- and high-dimensional features, along with seven state-of-the-art comparison algorithms, were subject to experiment. According to the empirical results in terms of both optimization and classification, it can be concluded that DTEA achieves outstanding performance on most of the datasets, with the proposed dynamic multi-tasking mechanism playing an effective role with all the essential components working together efficiently and cooperatively. In summary, from lower to higher dimensionality of features, the proposed algorithm DTEA generally achieves excellent performances in terms of the HV metric, owing to its dynamically adjusted balance between the convergence and diversity factors. However, DTEA may still encounter some insignificant setbacks in terms of the IGD and MCE metrics, which is probably because both IGD and MCE are more pertinent to evaluating the prominent convergence points (like knee points).

In future work, more kinds of evolutionary search tasks will be considered so as to adapt to more complex optimization environmental requirements. Moreover, there is also potential to study application of the proposed dynamic multi-tasking mechanism to many other subset selection problems, such as pattern mining, critical node detection, and neural network training. Application of the proposed algorithm in disease diagnosis and medical data analysis is also an indicated area of future work.