Enhancing Multi-Objective Optimization with Automatic Construction of Parallel Algorithm Portfolios

Abstract

It has been widely observed that there exists no universal best Multi-Objective Evolutionary Algorithm (MOEA) dominating all other MOEAs on all possible Multi-Objective Optimization Problems (MOPs). In this work, we advocate using the Parallel Algorithm Portfolio (PAP), which runs multiple MOEAs independently in parallel and gets the best out of them, to combine the advantages of different MOEAs. Since the manual construction of PAPs is non-trivial and tedious, we propose to automatically construct high-performance PAPs for solving MOPs. Specifically, we first propose a variant of PAPs, namely MOEAs/PAP, which can better determine the output solution set for MOPs than conventional PAPs. Then, we present an automatic construction approach for MOEAs/PAP with a novel performance metric for evaluating the performance of MOEAs across multiple MOPs. Finally, we use the proposed approach to construct an MOEAs/PAP based on a training set of MOPs and an algorithm configuration space defined by several variants of NSGA-II. Experimental results show that the automatically constructed MOEAs/PAP can even rival the state-of-the-art multi-operator-based MOEAs designed by human experts, demonstrating the huge potential of the automatic construction of PAPs in multi-objective optimization.

1. Introduction

Multi-Objective Optimization Problems (MOPs) are a class of classical optimization problems, which widely exist in many real-world applications such as finance [1], vehicle routing [2], and adversarial attack [3]. Multi-Objective Evolutionary Algorithms (MOEAs), as a kind of algorithm for solving MOPs, have attracted great attention over the past few decades and have shown a good performance [4,5,6,7,8,9,10]. Generally, MOEAs follow the generational framework where a population of solutions evolve from one generation to the next. In the evolution process, evolutionary operators (e.g., mutation and crossover) are used to generate offspring through the parents, while environmental selection operators are used to select elite individuals for the next-generation evolution.

Despite the tremendous success achieved by MOEAs, it has been widely observed [11,12,13,14] that there exists no universal best MOEA that dominates over all other MOEAs on all possible MOPs. Instead, different MOEAs are good at solving different MOPs due to their abilities on exploitation and exploration [15]. For example, MOEAs equipped with polynomial-based mutation (PM) [16] are good at searching in a local area, while those using simulated binary crossover (SBX) [17] and differential evolution (DE) [18,19] are capable of exploring the search space globally. Hence, in order to achieve a better overall performance for a diverse range of MOPs, it is natural and intuitive to combine the advantages of different MOEAs. One notable series of research efforts following this idea are multi-operator-based MOEAs [11,12,13,14], which adaptively allocate computational resources to MOEAs equipped with different operators when solving a MOP.

Apart from multi-operator-based MOEAs, and from a more general perspective of problem solving, there is an effective technique that exploits the complementarity between different algorithms by including them into a so-called algorithm portfolio (AP). To utilize an AP to solve a problem, Tang et al. [20,21] proposed a simple but effective strategy, called a parallel algorithm portfolio (PAP), that runs all member algorithms in the portfolio independently in parallel to obtain multiple solutions. Then, the best solution will be taken as the final output of the PAP. Although a PAP would consume more computational resources than a single algorithm, it has three important advantages. First, PAPs are easy-to-implement because they do not necessarily require any resource allocation since each member algorithm is simply assigned with the same amount of resource. Second, the performance of a PAP on any problem is the best performance achieved among its member algorithms on the problem. In other words, a PAP could achieve a much better overall performance than any of its member algorithms. Third, considering the tremendous growth of parallel computing architectures [22] (e.g., multi-core CPUs) over the last few decades, leveraging parallelism has become very important in designing effective solvers for hard optimization problems [23,24,25,26,27]. PAPs employ parallel solution strategies and thus allow for the use of modern computing facilities in an extremely simple way.

It is conceivable that any PAP’s effectiveness relies heavily on the diversity and complementarity among its member algorithms. In other words, the manual construction of high-quality PAPs is generally a challenging task, requiring domain experts (with a deep understanding of both algorithms and problems) to explore the vast design space of PAPs, which cannot be carried out manually with ease [28,29,30]. As an alternative, Tang and Liu [25,26] proposed a general framework, called automatic construction of PAPs, that seeks to automatically build PAPs by selecting the member algorithms from an algorithm configuration space, with the goal of optimizing the performance of the resulting PAP on a given problem set (called training set). Such a framework has been shown to be effective in building high-performance PAPs for combinatorial problems such as the Boolean Satisfiability Problem (SAT) [25], the Traveling Salesman Problem (TSP) [27,31], and the Vehicle Routing Problem (VRP) [26].

However, to the best of our knowledge, the potential of the automatic construction of PAPs has not been investigated in the area of multi-objective optimization. Considering its excellent performance on the above-mentioned problems and the practical significance of MOPs, studying how to utilize it to solve MOPs is thus valuable. In this work, we focus on automatically building PAPs for continuous MOPs. On the other hand, as a general framework, appropriately instantiating automatic PAP construction for a specific problem domain is non-trivial. Specifically, it requires careful designs of the algorithm configuration space and the performance metrics used in the construction process [26].

The main contributions of this work can be summarized as follows.

• Taking the characteristics of MOPs into account, we propose a novel variant form of PAP for MOPs, dubbed MOEAs/PAP. Its main difference from conventional PAPs lies in the method for determining the final output. MOEAs/PAP would compare the solution sets found by member algorithms and the solution set generated based on all the solutions found by member algorithms, and finally output the best solution set.
• We present an automatic construction approach for MOEAs/PAP with a novel metric that evaluates the performance of MOEAs/PAPs across multiple MOPs.
• Based on a training set of MOPs and an algorithm configuration space defined by several variants of NSGA-II, we use the proposed approach to construct an MOEAs/PAP, namely NSGA-II/PAP. Experimental results show that NSGA-II/PAP significantly outperforms existing single-operator-based MOEAs and the state-of-the-art multi-operator-based MOEAs designed by human experts. Such promising results indicate the huge potential of automatic construction of PAPs in multi-objective optimization.

The remainder of this paper is organized as follows. Section 2 presents the preliminaries and briefly reviews the related works. Section 3 gives the variant form of PAP for MOPs. The automatic construction approach, as well as the algorithm configuration space, the training set, and the performance metric used in construction, are presented in Section 4. Section 5 presents the experimental study. Finally, Section 6 concludes the paper.

2. Preliminaries and Related Work

2.1. Multi-Objective Optimization Problems

Without loss of generality, in this work we assume optimization problems take the minimization form. MOPs are optimization problems with multiple objectives, defined as follows:

$$\begin{array}{cc}\hfill minF\left(\mathbf{x}\right)& =[f_1\left(\mathbf{x}\right),f_2\left(\mathbf{x}\right),\cdots ,f_m\left(\mathbf{x}\right)]\in \Pi \hfill \\ \hfill \mathbf{x}& =[x_1,x_2,\cdots ,x_n]\in D\hfill \end{array},$$

(1)

where $\mathbf{x}$ is the decision vector, which consists of n decision variables. The objective vector $F:D\to \Pi$ consists of m objectives; $D\subseteq R^n$ and $\Pi \subseteq R^m$ denote the decision space and the objective space, respectively. The objectives in Equation (1) are often in conflict with each other. That is, the improvement of one objective may lead to the deterioration of another. This gives rise to a set of optimal solutions (largely known as Pareto-optimal solutions), instead of a single optimal solution. The concept of Pareto-optimal is defined as follows.

Definition 1. 

Given two decision vectors $\mathbf{u},\mathbf{v}$ and their corresponding objective vectors $F\left(\mathbf{u}\right),F\left(\mathbf{v}\right)$. $\mathbf{u}$ dominates $\mathbf{v}$ (denoted as $\mathbf{u}\prec \mathbf{v}$), if and only if $\forall i\in \{1,\cdots ,m\},f_i\left(\mathbf{u}\right)\le f_i\left(\mathbf{v}\right)$ and $\exists i\in \{1,\cdots ,m\},f_i\left(\mathbf{u}\right)<f_i\left(\mathbf{v}\right)$.

Definition 2. 

A solution ${\mathbf{x}}^{*}$ is Pareto-optimal if and only if there exists no $\mathbf{u}\in \Omega$ such that $\mathbf{u}\prec {\mathbf{x}}^{*}$. The set of all Pareto-optimal solutions is called the Pareto set. The corresponding objective vector set of the Pareto set is called the Pareto front.

There exists a large body of benchmark sets for MOPs. Among them, Zizler–Deb–Thiele (ZDT) [32], Deb–Thiele–Laumanns–Zizler (DTLZ) [33], walking-fish-group (WFG) [34], and CEC 2009 competitions (UF) [35] are commonly used in the literature. These problems are characterized by multimodality, co-replicated Pareto sets and Pareto fronts, separability between decision variables, and correlation between decision variables and objectives, deception, and epistasis [34]. These are of varying difficulty and complexity and cover a variety of real-world problems. Moreover, the commonly used metrics for evaluating the solutions on MOPs include Inverted Generational Distance (IGD) [36] and Hypervolume (HV) [37,38].

2.2. Multi-Objective Evolutionary Algorithms

There is a large body of MOEAs, such as NSGA-II [4], MOEA/D [5], SPEA2 [8], PAES [10], and EMOCA [39]. This article will mainly introduce Genetic Algorithms (GAs) [4,5] and Differential Evolution (DE) Algorithms [40], and propose our algorithm based on these. GA is based on biological inspiration and generates new offspring by imitating the crossover and mutation process in biological genetic evolution. One of the most classic and widely used variation strategies is composed of a simulated binary crossover (SBX) operator and a polynomial mutation (PM) operator [41]. Specifically, supposing the two parent individuals are ${\mathbf{x}}^1=(x_1^1,x_2^1,\cdots ,x_n^1)$ and ${\mathbf{x}}^2=(x_1^2,x_2^2,\cdots ,x_n^2)$, and the generated offspring are ${\mathbf{c}}^1=(c_1^1,c_2^1,\cdots ,c_n^1)$ and ${\mathbf{c}}^2=(c_1^2,c_2^2,\cdots ,c_n^2)$. SBX and PM operate as follows.

SBX:

$$\begin{array}{cc}\hfill c_i^1& =0.5\times [(1+\beta )·x_i^1+(1-\beta )·x_i^2]\hfill \\ \hfill c_i^2& =0.5\times [(1-\beta )·x_i^1+(1+\beta )·x_i^2]\hfill \end{array},$$

(2)

where $\beta$ is a parameter, defined as follows:

$$\beta =\left\{\begin{array}{cc}{(r\times 2)}^{1/(1+\eta )}\hfill & \mathrm{if}r\le 0.5\hfill \\ {(1/(2-r\times 2))}^{1/(1+\eta )}\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(3)

where r is a random value within $[0,1]$ and $\eta$ is a parameter representing the similarity between the offspring individual and the parent individual. The larger the value, the higher the similarity.

PM:

$$\begin{array}{c}\hfill c_i^1=x_i^1+\Delta ·(u_i-l_i)\end{array},$$

(4)

$\Delta$ is defined as follows:

$$\Delta =\left\{\begin{array}{cc}{(2r+(1-2r){\delta }_1^{\eta +1})}^{1/(\eta +1)-1}-1\hfill & \mathrm{if}r\le 0.5\hfill \\ {((4r-1)+(2r-1){\delta }_2^{\eta +1})}^{1/(\eta +1)}\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(5)

where ${\delta }_1=(u_i-x_i^1)/(u_i-l_i)$ and ${\delta }_2=(x_i^1-l_i)/(u_i-l_i)$, r is a random parameter within [0, 1], $u_i$ and $l_i$ are the upper and lower bounds of the i-th dimension variable, respectively. $\eta$ is a parameter that representing the similarity between the offspring individual and the parent individual. The larger the value, the higher the similarity.

rand/p:

$$c_j^i=\left\{\begin{array}{cc}{x}_j^{r_3}+F·\sum _{l=1}^p(x_j^{r_1,l}-x_j^{r_2,l})\hfill & \mathrm{if}r\le CR\mathrm{or}j=j_r\hfill \\ x_j^i,\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(6)

best/p:

$$c_j^i=\left\{\begin{array}{cc}{x}_j^{best}+F·\sum _{l=1}^p(x_j^{r_1,l}-x_j^{r_2,l}),\hfill & \mathrm{if}r\le CR\mathrm{or}j=j_r\hfill \\ x_j^i,\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(7)

current-to-rand/p:

$$c_j^i=\left\{\begin{array}{cc}{x}_j^i+K·(x_j^{r_3}-x_j^i)+F·\sum _{l=1}^p(x_j^{r_1,l}-x_j^{r_2,l})\hfill & \mathrm{if}r\le CR\mathrm{or}j=j_r\hfill \\ x_j^i\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(8)

current-to-best/p:

$$c_j^i=\left\{\begin{array}{cc}{x}_j^i+K·(x_j^{best}-x_j^i)+F·\sum _{l=1}^p(x_j^{r_1,l}-x_j^{r_2,l}),\hfill & \mathrm{if}r\le CRorj=j_r\hfill \\ x_j^i,\hfill & \mathrm{otherwise}\hfill \end{array}\right..$$

(9)

Different from GAs, DEs mainly perform gradient estimation through differential mutation operators to generate offspring. Over the past few decades, various types of differential mutation operators have been proposed. Mezura-Montes et al. [40] summarized them in Equations (6)–(9):

In Equations (6)–(9), p is the number of pairs of solutions used to compute the differences in the mutation operator, $c_j^i$ is the j-th dimension variable of the i-th offspring, $x_j^{r_3}$ is the j-th dimension variable of the donor solution chosen at random, $x_j^{best}$ is the j-th dimension variable of the best solution in the population as the donor solution, $x_j^i$ is the j-th dimension variable of the current parent, $x_j^{r_1,l}$ and $x_j^{r_2,l}$ are the j-th dimension variables of the p-th pair to compute the mutation differential, and F, K, and $CR$ are parameters in the DE algorithm.

2.3. Multi-Operator-Based MOEAs

Multi-Operator MOEAs mainly adaptively allocate the computation resources (population sizes) among different operators. The allocation strategy could be designed by human experts [11,12] or learned automatically [13,14]. Specifically, motivated by the well-known ensemble learning approach AdaBoost [42], Wang et al. [11] proposed to use the accumulated data of previous generations to adjust the population size of each operator in the next generation, such that the operators that produce better solutions in the previous generation will occupy more computational resources. Gao et al. [12] took the solutions generated by multiple operators as a whole and determined the local search strategy, global search direction, and final convergence direction by analyzing the features in the entire target space. Elsayed et al. [13] constructed a solver that includes two complementary algorithms. The inferior algorithm optimizes itself by learning the strategy of the superior algorithm and gives feedback to finetune the superior algorithm. Finally, the desired result is output from the high-quality algorithm. Sun et al. [14] regarded the parameter adaptation process as a Markov Decision Process (MDP). Through reinforcement learning, different parameter controllers are learned for different problems, such that they have different adaptive parameter selection methods for these problems. All of the above methods need a well-designed adaptive method to allocate computational resources to the member algorithms.

2.4. PAPs and Automatic Construction of PAPs

Essentially, a PAP [27,31,43] is a set of member algorithms that run in parallel:

$$\begin{array}{c}\hfill P=\left\{{\theta }_1,{\theta }_2,\cdots ,{\theta }_k\right\}\end{array},$$

(10)

where P is the PAP solver, ${\theta }_i$ is the i-th member algorithm of P, and k is the number of member algorithms.

Let $Metric(\theta ,z)$ denote the performance of an algorithm $\theta$ on a problem z (e.g., an MOP) in terms of the performance metric $Metric$ (e.g., HV for MOPs). Then, the performance of P on a problem z, denoted as $\Omega (P,z)$, is the best performance achieved among the member algorithms of P on z:

$$\Omega (P,z)=\underset{\theta \in P}{max}Metric(\theta ,z).$$

(11)

Without loss of generality, we assume for M, the larger the better. Let Z denote a set of problems. The performance of P on Z, denoted as $\Omega (P,Z)$, is an aggregated value of the performance of P on all the problems in Z:

$$\begin{array}{c}\hfill \Omega (P,Z)=\frac{1}{\left|Z\right|}·\sum _{z\in Z}\Omega (P,z).\end{array}$$

(12)

The conventional way to construct P is to manually choose member algorithms for it, as shown in [20,21], which is, however, non-trivial and tedious. To address this issue, Tang and Liu [25,26] proposed the framework of the automatic construction of PAPs. Specifically, given a performance metric $Metric$, a training set Z, and an algorithm configuration space $\Theta$, the goal is to find at most k algorithms from $\Theta$ to form a PAP $P^{*}$, which has the best performance on Z in terms of M:

$$\begin{array}{c}\hfill P^{*}=arg{max}_{P\subseteq \Theta \&\left|P\right|\le k}\Omega (P,Z),\end{array}$$

(13)

where $\Omega (P,Z)$ is defined in Equation (12). The results in [25,26,27,31] have shown that the PAPs automatically built by solving the problem in Equation (13) could achieve excellent performance on problems including SAT, TSPs, and VRPs, which are even better than the algorithms designed by human experts.

3. MOEAs/PAP for MOPs

As stated in Equation (11), a PAP would return the best solution among the ones found by its member algorithms. However, this could be problematic in multi-objective optimization since an MOEA outputs a set of Pareto-optimal solutions, instead of a single one, for a given MOP. One way to mitigate this issue is to compare the solution sets found by the member algorithms in terms of the performance metric (e.g., HV), and finally returns the best solution set.

On the other hand, it is likely that the solution sets found by member algorithms contain solutions that do not dominate each other. Thus, we propose a novel procedure called Restructure to obtain the final solution set. Specifically, when all member algorithms have completed their execution, the Pareto-optimal solution sets obtained by them are first combined. Assuming each member algorithm has a population size of P, the size of this set is $N·P$ (N is the number of member algorithms). Then, Restructure employs the non-dominated sorting mechanism of NSGA-II [4] on these solutions and finally selects the top-P solutions to form the output solution set. Finally, all the solution sets found by member algorithms, as well as the one generated by Restructure, are compared, and the best set is returned as the output of MOEAs/PAP, as illustrated in Figure 1.

Formally, for an MOEAs/PAP, denoted as P, its performance on an MOP z can be described as follows:

$$\Omega (P,z)=max\{\underset{\theta \in P}{max}Metric(\theta ,z),Metric(\overline{\theta },z)\},$$

(14)

where $\overline{\theta }$ represents the Restructure procedure. Note Equation (14) is slightly different from Equation (11) (i.e., the performance of conventional PAPs) due to the Restructure procedure.

4. Automatic Construction of MOEAs/PAP

As aforementioned, the member algorithms of a PAP could be automatically determined by solving the problem defined in Equation (13). Below, we first introduce the algorithm configuration space $\Theta$, the training set Z, and the performance metric $Metric$; then we present the automatic construction approach for MOEAs/PAP.

4.1. Algorithm Configuration Space $\Theta$ and Training Set Z

The algorithm configuration space $\Theta$ is defined by a number of parameterized algorithms $B_1,B_2,\dots ,B_c$ (called foundation algorithms). Each foundation algorithm $B_i$ has a number of parameters, whose values control the behavior of $B_i$ and thus can largely affect its performance. Therefore, $B_i$ taking different parameter values can actually be considered as different algorithms. Let ${\Theta }_i$ denote the set of all unique algorithms obtained by taking all possible values of the parameters of $B_i$. Then, the algorithm configuration space $\Theta$ is the union of ${\Theta }_1,{\Theta }_2,\dots ,{\Theta }_c$, i.e., $\Theta ={\Theta }_1\cup {\Theta }_2\cup \dots \cup {\Theta }_c$. In this work, the foundation algorithms are parameterized MOEAs, i.e., NSGA-II with different evolutionary operators. The training set Z is a set of problems (MOPs), and should be representative of the target problems to which the constructed MOEAs/PAP is expected to be applied. The details of the algorithm configuration space $\Theta$ and the training set Z are given in the experiments (see Section 5.1 and Section 5.2).

4.2. Performance Metric

As stated in Equation (12), during the construction process, the performance of the PAP on the training set Z is an aggregated value of its performance on all the problems in Z. However, for MOPs, the commonly used IGD and HV cannot be directly used as the performance metric $Metric$ here. The main reason is that, for these two metrics, the performance of an MOEA on different MOPs cannot be directly aggregated (or compared) due to the different scales.

Hence, we propose to re-scale HV values within $[0,1]$, by using the following HV Ratio (HVR):

$$hvr=\frac{hv}{h{v}^{*}},$$

(15)

where $hv$ is the HV value of the solution set found by an MOEA on an MOP, and $h{v}^{*}$ is the HV value of the optimal solution set of the problem. Both $hv$ and $h{v}^{*}$ are calculated based on the same reference point in the objective space, which is obtained by taking the maximum on each objective achieved among the solution set found by the MOEA. However, there is still an issue in the actual use of HVR. During the construction process of MOEAs/PAP, there is a large body of MOEAs, which would be evaluated. Since the reference point is up to the solution set found by the evaluated MOEA, given an MOP, $h{v}^{*}$ needs to be recalculated for every evaluated MOEA. Considering the size of the optimal solution set is usually very large (typically larger than 1000), thus in the construction process it requires a large amount of calculations for $h{v}^{*}$.

To address the above issue, we propose to use a fixed reference point for a given MOP, such that $h{v}^{*}$ needs to be calculated for the problem for only once. One way is to use the the reference point taking the upper bounds (note that we assume MOPs take the minimization form) of the objective values. However, this will cause the gap between $hv$ and $h{v}^{*}$ to be very small, in which case $hvr$ tends to be 1 and loses its evaluation ability. Therefore, we calculate the HVs of the undominated space for the found solution set and the optimal solution set, and finally use the ratio of them to assess the performance of an MOEA on a MOP.

The new performance metric is named Inverted Hypervolume Ratio (IHVR):

$$ihvr=\frac{h{v}_{all}-h{v}^{*}}{h{v}_{all}-hv},$$

(16)

where $hv$ and $h{v}^{*}$ are calculated based on the reference point taking the upper bounds of the objective values, and $h{v}_{all}$ is the HV of the cube formed by the ranges of objective values of the MOP. Thus, $h{v}_{all}-hv$ and $h{v}_{all}-h{v}^{*}$ are the HVs of the undominated space for the solution set found by the evaluated MOEA and the optimal solution set, respectively. Note that $ihvr\in (0,1]$, and a larger value of $ihvr$ is better. IHVR can distinguish well between MOEAs with a different performance, and can also save a lot of calculations. In this work, IHVR is used as the performance metric $Metric$.

4.3. Automatic Construction Approach

Algorithm 1 presents the automatic construction approach for MOEAs/PAP. Starting from an empty set (line 2), the approach constructs the PAP (denoted as P) iteratively. Specifically, each iteration of the approach (lines 3–15) consists of two subsequent phases. In the first phase, an existing automatic algorithm configuration tool, namely SMAC 3 [44], is used to search in $\Theta$ to find the algorithm that can improve the performance of the current PAP to the largest extent (line 5), and then this algorithm is inserted into P (line 6). This phase is similar to the commonly-used greedy approach in the automatic construction of PAPs [27,31]. Additionally, we introduce a new phase, namely simplification, as the second phase in Algorithm 1. In this phase (lines 7–14), P would be simplified by removing the member algorithms that do not contribute at all to its performance (meaning removing these algorithms has no effect on the performance of P on the training set). Considering the size of P is bounded (line 3), removing the redundant algorithms from P is meaningful because this will leave space for new member algorithms that can improve the performance of P. Finally, the approach would be terminated if the maximum number of member algorithms is reached (line 3), or the performance of P does not improve with the inclusion of the found algorithm (line 6).

Algorithm 1: Automatic construction of MOEAs/PAP.

input:  Algorithm configuration space $\Theta$, training set Z, maximum number of member algorithms k   output:  the final PAP P $\phantom{(}$

5. Experiments

In the experiments, we first used the proposed automatic construction approach to build a MOEAs/PAP with a training set, and then compared it with other MOEAs on the testing set to verify its performance. Finally, we analyzed the performance of each member algorithm of the MOEAs/PAP, as well as the impact of the Restructure procedure.

5.1. Benchmark Sets

We collected the commonly-used MOP benchmarks in the literature, including ZDT [32], DTLZ [33], WFG [34], and UF [35]. These benchmark sets contain 32 MOPs in total, as summarized in

Table 1. The problem benchmark sets used in the experiments.

Problem	Decision Vector Dimension $\left(\mathit{n}\right)$	Objective Vector Dimension $\left(\mathit{m}\right)$
ZDT1-3	30	2
ZDT4	10	2
ZDT5	11	2
ZDT6	10	2
DTLZ1-7	11	2
WFG1-9	12	3
UF1-7	30	2
UF8-10	30	3

. From each of these benchmark sets, several problems were chosen as the training problems, and the remaining problems were used for testing. Specifically, the training set includes 17 MOPs in total, i.e., ZDT3-6, DTLZ2-5, WFG1, WFG5-6, WFG9, UF3-4, UF6-7, and UF10; the testing set contains 15 MOPs in total, i.e., ZDT1-2, DTLZ1, DTLZ6-7, WFG2-4, WFG7-8, UF1-2, UF5, and UF8-9.

5.2. Construction of NSGA-II/PAP

To use the approach in Algorithm 1 to construct an MOEAs/PAP, one needs to provide a training set of MOPs (detailed in Section 5.1) and an algorithm configuration space. Here, the algorithm configuration space is defined based on five variants of NSGA-II [4], each of which adopts a different parameterized evolutionary operator as defined in Equations (2)–(9). All the foundation algorithms, as well as their evolutionary operators and parameters, are summarized in

Table 2. The five foundation algorithms (variants of NSGA-II), as well as their evolutionary operators and parameters, that define the algorithm configuration space. For brevity, each foundation algorithm is denoted by the used evolutionary operator. n represents the decision vector dimension of the problem.

Foundation Algorithm	Parameter	Value Range
SBX + PM	$\eta$ of SBX	$\{1,2,\cdots ,100\}$
	$p_c$	$\left\{1\right\}$
	$\eta$ of PM	$\{1,2,\cdots ,100$}
	$p_m$	$\{1/n\}$
rand/p	F	$\left(0,2\right]$
	p	$\{1,2\}$
	$CR$	$\left(0,1\right]$
best/p	F	$\left(0,2\right]$
	p	$\{1,2\}$
	$CR$	$\left(0,1\right]$
current-to-rand/p	F	$\left(0,2\right]$
	K	$\left(0,1\right]$
	p	$\left\{1\right\}$
	$CR$	$\left(0,1\right]$
current-to-best/p	F	$\left(0,2\right]$
	K	$\left(0,1\right]$
	p	$\left\{1\right\}$
	$CR$	$\left(0,1\right]$

. Finally, the maximum number of member algorithms, i.e., k, was set to 10, considering that 10-core machines are widely available now.

Given the training set and the algorithm configuration space, we used the proposed approach to construct an MOEAs/PAP. Since all the foundation algorithms are variants of NSGA-II, the constructed MOEAs/PAP is named NSGA-II/PAP. Specifically, NSGA-II/PAP contains six member algorithms, which are detailed in

Table 3. Member algorithms of the constructed NSGA-II/PAP. For brevity, each member algorithm is denoted by the used evolutionary operator.

Member Algorithm	Parameter
SBX + PM	$\eta$ of SBX $=81$, $\eta$ of PM $=18$
	$p_c=1,p_m=1/n$
best/p	$p=2,F=1.16198$
	$CR=0.07724$
rand/p	$p=1,F=0.07578$
	$CR=0.41910$
best/p	$p=2,F=0.61077$
	$CR=0.57930$
SBX + PM	$\eta$ of SBX $=67$, $\eta$ of PM $=54$
	$p_c=1,p_m=1/n$
SBX + PM	$\eta$ of SBX $=45$, $\eta$ of PM $=23$
	$p_c=1,p_m=1/n$

.

5.3. Compared Algorithms and Experimental Protocol

We compare NSGA-II/PAP with the NSGA-II-based MOEA including the original NSGA-II [4], its variant using the DE operator, namely NSGA-II-DE [18], and the state-of-the-art multi-operator-based MOEAs, dubbed NSGA-II/MOE [11]. Also, we included other types of MOEAs, MOPSO [9] and MOEA/D [5], in the comparison. The experimental results in [11] have shown that NSGA-II/MOE could consistently achieve a better performance than conventional MOEAs on a large body of benchmark sets.

For the compared algorithms, the recommended parameter settings in the original publication were used, which are listed in

Table 4. Parameter settings of the compared algorithms.

Algorithm	Operator	Parameter
NSGA-II	SBX + PM	$\eta$ of SBX $=20$, $\eta$ of PM $=20$
		$p_m=1/n$
NSGA-II-DE	rand/1	$F=0.5,CR=0.5$
NSGA-II/MOE	rand/1	$F=0.5,CR=1.0$
	rand/2	$F=0.5,CR=1.0$
	current-to-rand/1	$F=0.5,CR=1.0,K=0.5$
	SBX + PM	$\eta$ of SBX $=20$, $\eta$ of PM $=20$
		$p_m=1/n$
MOEA/D	SBX + PM	$\eta$ of SBX $=20$, $\eta$ of PM $=20$
		$p_m=1/n,T=10$
MOEA/D	-	$C1=20,C2=20$
		$p_m=1/n,W=0.4$

. For all the tested algorithms, the population sizes and the number of fitness evaluations (FEs) were kept the same. Specifically, for UF1-7, population size was set to 100 and the max generation number was set to 500; for UF8-10, population size was set to 500 and the max generation number was set to 600; for WFG1-9, population size was set to 500 and the max generation number was set to 250; for all DTLZ and ZDT, population size was set to 100 and the max generation number was set to 250.

Since NSGA-II/PAP runs its member algorithms in parallel, it would perform more FEs than the compared algorithms. Hence, in the experiments, for each compared algorithm, we also considered two variants of it that performed the same number of FEs as NSGA-II/PAP. Let N be the number of member algorithms of NSGA-II/PAP (here, N = 6). Specifically, the first variant, indicated by Ngen, increased the maximum generation number of the algorithm to N times the original number. The second variant, indicated by Nsize, increased the population size to N times the original population size, and used the Restructure procedure (see Section 3) to choose the final Pareto-optimal solution set of the original population size from the final population. We implemented all the tested algorithms with Python (version 3.8) library Geatpy (version 2.6.0 (https://github.com/geatpy-dev/geatpy) (accessed on 31 October 2022)). All the experiments were conducted on an Intel Xeon machine with 96 GB RAM and 16 cores (2.10 GHz, 20 MB Cache), running Ubuntu 16.04.

For each testing problem, each tested algorithm would be independently applied for 30 times; the average and standard deviation of the achieved HV and IGD would be reported.

5.4. Testing Results and Analysis

The testing results in terms of HV are presented in

Table 5. The mean ± standard deviation of the HV values across the 30 runs over the testing set. For each testing problem, the best performance is indicated in bold (note for HV, the larger, the better). † indicates that the performance of the algorithm is significantly different from the performance of NSGA-II/PAP (according to a Wilcoxon’s bilateral rank sum test with $p=0.05$). The significance test of NSGA-II/PAP against the compared algorithm is summarized in the win-draw-loss (W-D-L) counts.

Problem	NSGA-II/PAP	NSGA-II (Nsize)	NSGA-II (Ngen)	NSGA-II-DE (Nsize)	NSGA-II-DE (Ngen)	NSGA-II/MOE (Nsize)	NSGA-II/MOE (Ngen)
UF1	0.6770 ± 4.50 × 10${}^{-4}$	0.6147 ± 1.01 × 10${}^{-4}$ †	0.6173 ± 3.64 × 10${}^{-4}$ †	0.6283 ± 2.44 × 10${}^{-5}$ †	0.6469 ± 1.83 × 10${}^{-4}$ †	0.6574 ± 2.11 × 10${}^{-4}$ †	0.6920 ± 5.54 × 10${}^{-4}$†
UF2	0.6957 ± 1.09 × 10${}^{-5}$	0.6814 ± 3.80 × 10${}^{-5}$ †	0.6889 ± 5.04 × 10${}^{-5}$ †	0.6749 ± 8.27 × 10${}^{-5}$ †	0.6947 ± 2.74 × 10${}^{-5}$	0.6814 ± 1.68 × 10${}^{-4}$ †	0.7003 ± 2.90 × 10${}^{-5}$†
UF5	0.2503 ± 2.17 × 10${}^{-3}$	0.1944 ± 7.70 × 10${}^{-4}$ †	0.1755 ± 2.08 × 10${}^{-3}$ †	0.2129 ± 1.20× 10${}^{-13}$ †	0.1979 ± 1.32 × 10${}^{-3}$ †	0.2041 ± 1.45 × 10${}^{-3}$ †	0.1576 ± 4.19 × 10${}^{-3}$ †
UF8	0.4394 ± 1.08 × 10${}^{-3}$	0.2827 ± 1.67 × 10${}^{-4}$ †	0.3142 ± 9.92 × 10${}^{-4}$ †	0.2992 ± 1.19 × 10${}^{-4}$ †	0.2794 ± 1.34 × 10${}^{-4}$ †	0.4157 ± 1.19 × 10${}^{-3}$ †	0.4252 ± 4.08 × 10${}^{-3}$
UF9	0.7619 ± 1.92 × 10${}^{-4}$	0.7014 ± 9.52 × 10${}^{-4}$ †	0.6452 ± 5.52 × 10${}^{-3}$ †	0.7254 ± 1.92 × 10${}^{-7}$ †	0.7127 ± 5.61 × 10${}^{-6}$ †	0.5276 ± 1.31 × 10${}^{-2}$ †	0.4190 ± 1.59 × 10${}^{-2}$ †
WFG2	0.9425 ± 1.13 × 10${}^{-7}$	0.9393 ± 6.09 × 10${}^{-7}$ †	0.9410 ± 1.17 × 10${}^{-7}$ †	0.9390 ± 4.45 × 10${}^{-7}$ †	0.9392 ± 2.12 × 10${}^{-7}$ †	0.9406 ± 3.33 × 10${}^{-7}$ †	0.9408 ± 1.49 × 10${}^{-7}$ †
WFG3	0.6539 ± 5.95 × 10${}^{-6}$	0.6439 ± 6.25 × 10${}^{-6}$ †	0.6395 ± 8.55 × 10${}^{-6}$ †	0.6332 ± 4.57 × 10${}^{-6}$ †	0.6119 ± 1.58 × 10${}^{-5}$ †	0.6439 ± 3.06 × 10${}^{-6}$ †	0.6350 ± 1.05 × 10${}^{-5}$ †
WFG4	0.5745 ± 1.43 × 10${}^{-6}$	0.5714 ± 6.99 × 10${}^{-7}$ †	0.5651 ± 3.08 × 10${}^{-6}$ †	0.5490 ± 1.73 × 10${}^{-6}$ †	0.5372 ± 3.02 × 10${}^{-6}$ †	0.5687 ± 1.10 × 10${}^{-6}$ †	0.5583 ± 4.28 × 10${}^{-6}$ †
WFG7	0.1077 ± 3.61 × 10${}^{-6}$	0.1027 ± 2.88 × 10${}^{-6}$ †	0.1114 ± 4.64 × 10${}^{-7}$†	0.0276 ± 2.07 × 10${}^{-5}$ †	0.0731 ± 3.85 × 10${}^{-5}$ †	0.1025 ± 2.66 × 10${}^{-6}$ †	0.1038 ± 8.77 × 10${}^{-6}$ †
WFG8	0.2121 ± 5.88 × 10${}^{-6}$	0.2261 ± 4.72 × 10${}^{-7}$ †	0.2450 ± 2.21 × 10${}^{-7}$†	0.0775 ± 1.92 × 10${}^{-5}$ †	0.1360 ± 9.81 × 10${}^{-5}$ †	0.2172 ± 5.12 × 10${}^{-7}$ †	0.2378 ± 1.02 × 10${}^{-5}$ †
DTLZ1	0.5556 ± 5.79 × 10${}^{-4}$	0.5773 ± 9.67 × 10${}^{-7}$ †	0.5812 ± 1.36 × 10${}^{-7}$ †	0.0000 ± 0.00 × 10${}^0$ †	0.5816 ± 3.20 × 10${}^{-8}$†	0.5581 ± 1.02 × 10${}^{-2}$ †	0.4668 ± 5.18 × 10${}^{-2}$ †
DTLZ6	0.3465 ± 2.52 × 10${}^{-8}$	0.3451 ± 2.09 × 10${}^{-7}$ †	0.3462 ± 6.18 × 10${}^{-8}$ †	0.3451 ± 2.74 × 10${}^{-7}$ †	0.3463 ± 5.14 × 10${}^{-8}$ †	0.3448 ± 3.00 × 10${}^{-7}$ †	0.3468 ± 6.28 × 10${}^{-8}$†
DTLZ7	0.2428 ± 1.23 × 10${}^{-9}$	0.2420 ± 4.04 × 10${}^{-8}$ †	0.2405 ± 1.43 × 10${}^{-4}$ †	0.2420 ± 6.57 × 10${}^{-8}$ †	0.2425 ± 2.94 × 10${}^{-9}$ †	0.2418 ± 8.32 × 10${}^{-8}$ †	0.2383 ± 2.77 × 10${}^{-4}$ †
ZDT1	0.7198 ± 3.84 × 10${}^{-8}$	0.7164 ± 1.02 × 10${}^{-6}$ †	0.7195 ± 5.31 × 10${}^{-8}$ †	0.7164 ± 6.66 × 10${}^{-7}$ †	0.7198 ± 2.83 × 10${}^{-8}$	0.7152 ± 1.61 × 10${}^{-6}$ †	0.7188 ± 1.70 × 10${}^{-7}$ †
ZDT2	0.4442 ± 3.29 × 10${}^{-8}$	0.4414 ± 3.66 × 10${}^{-7}$ †	0.4436 ± 6.48 × 10${}^{-8}$ †	0.4409 ± 6.24 × 10${}^{-7}$ †	0.4444 ± 1.52 × 10${}^{-8}$†	0.4403 ± 1.19 × 10${}^{-6}$ †	0.4438 ± 1.27 × 10${}^{-7}$ †
W-D-L	-	13-0-2	12-0-3	15-0-0	11-2-2	13-0-2	10-1-4

and

Table 6. The mean ± standard deviation of the HV values across the 30 runs over the testing set. For each testing problem, the best performance is indicated in bold (note for HV, the larger, the better). † indicates that the performance of the algorithm is significantly different from the performance of NSGA-II/PAP (according to a Wilcoxon’s bilateral rank sum test with $p=0.05$). The significance test of NSGA-II/PAP against the compared algorithm is summarized in the win-draw-loss (W-D-L) counts.

Problem	NSGA-II/PAP	MOEA/D (Ngen)	MOPSO (Ngen)	MOEA/D (Nsize)	MOPSO (Nsize)
UF1	0.6770 ± 4.50 × 10${}^{-4}$	0.6662 ± 4.52 × 10${}^{-4}$ †	0.6708 ± 5.96 × 10${}^{-4}$ †	0.6873 ± 1.91 × 10${}^{-4}$†	0.6784 ± 6.74 × 10${}^{-4}$
UF2	0.6957 ± 1.09 × 10${}^{-5}$	0.6972 ± 1.30 × 10${}^{-4}$	0.6997 ± 3.13 × 10${}^{-6}$ †	0.7130 ± 8.17 × 10${}^{-5}$†	0.7111 ± 9.78 × 10${}^{-7}$ †
UF5	0.2503 ± 2.17 × 10${}^{-3}$	0.1610 ± 4.70 × 10${}^{-3}$ †	0.1088 ± 4.64 × 10${}^{-3}$ †	0.2223 ± 2.45 × 10${}^{-3}$ †	0.0900 ± 2.69 × 10${}^{-3}$ †
UF8	0.4394 ± 1.08 × 10${}^{-3}$	0.4545 ± 2.75 × 10${}^{-3}$ †	0.4602 ± 2.97 × 10${}^{-3}$ †	0.4847 ± 9.02 × 10${}^{-4}$ †	0.5074 ± 7.02 × 10${}^{-4}$†
UF9	0.7619 ± 1.92 × 10${}^{-4}$	0.7504 ± 2.65 × 10${}^{-3}$ †	0.5970 ± 1.17 × 10${}^{-2}$ †	0.7554 ± 2.07 × 10${}^{-4}$ †	0.7416 ± 3.71 × 10${}^{-3}$ †
WFG2	0.9425 ± 1.13 × 10${}^{-7}$	0.9364 ± 1.86 × 10${}^{-6}$ †	0.9287 ± 4.76 × 10${}^{-7}$ †	0.9411 ± 1.44 × 10${}^{-7}$ †	0.9427 ± 4.49 × 10${}^{-8}$
WFG3	0.6539 ± 5.95 × 10${}^{-6}$	0.6417 ± 6.74 × 10${}^{-6}$ †	0.5362 ± 1.05 × 10${}^{-4}$ †	0.6490 ± 5.13 × 10${}^{-7}$ †	0.6290 ± 1.57 × 10${}^{-5}$ †
WFG4	0.5745 ± 1.43 × 10${}^{-6}$	0.5768 ± 1.06 × 10${}^{-6}$	0.5046 ± 1.02 × 10${}^{-5}$ †	0.5905 ± 2.02 × 10${}^{-7}$†	0.5622 ± 9.02 × 10${}^{-7}$ †
WFG7	0.2121 ± 5.88 × 10${}^{-6}$	0.0976 ± 5.07 × 10${}^{-7}$ †	0.0106 ± 1.37 × 10${}^{-5}$ †	0.1089 ± 1.74 × 10${}^{-7}$ †	0.0160 ± 1.51 × 10${}^{-5}$ †
WFG8	0.5556 ± 5.79 × 10${}^{-4}$	0.2373 ± 2.12 × 10${}^{-7}$ †	0.0627 ± 1.89 × 10${}^{-7}$ †	0.2384 ± 1.84 × 10${}^{-7}$ †	0.0659 ± 1.32 × 10${}^{-7}$ †
DTLZ1	0.5556 ± 5.79 × 10${}^{-4}$	0.5787 ± 6.39 × 10${}^{-7}$ †	0.0000 ± 0.00 × 10${}^0$ †	0.5854 ± 1.75 × 10${}^{-8}$†	0.0000 ± 0.00 × 10${}^0$ †
DTLZ6	0.3465 ± 2.52 × 10${}^{-8}$	0.3438 ± 1.12 × 10${}^{-6}$ †	0.3460 ± 2.34 × 10${}^{-7}$	0.3497 ± 4.04 × 10${}^{-8}$ †	0.3498 ± 2.74 × 10${}^{-9}$†
DTLZ7	0.2428 ± 1.23 × 10${}^{-9}$	0.2412 ± 7.92 × 10${}^{-7}$ †	0.2419 ± 1.61 × 10${}^{-7}$ †	0.2433 ± 2.07 × 10${}^{-8}$	0.2430 ± 5.57 × 10${}^{-10}$
ZDT1	0.7198 ± 3.84 × 10${}^{-8}$	0.7167 ± 6.31 × 10${}^{-7}$ †	0.7149 ± 6.57 × 10${}^{-7}$ †	0.7132 ± 5.01 × 10${}^{-8}$ †	0.7129 ± 7.33 × 10${}^{-9}$ †
ZDT2	0.4442 ± 3.29 × 10${}^{-8}$	0.4413 ± 6.20 × 10${}^{-7}$ †	0.4422 ± 4.95 × 10${}^{-7}$ †	0.4427 ± 2.48 × 10${}^{-8}$ †	0.4435 ± 6.31 × 10${}^{-8}$ †
W-D-L	-	11-2-2	12-1-2	8-1-6	9-3-3

. And the results in terms of IGD are presented in

Table 7. The mean ± standard deviation of the IGD values across the 30 runs over the testing set. For each testing problem, the best performance is indicated in bold (note for IGD, the smaller, the better). † indicates that the performance of the algorithm is significantly different from the performance of NSGA-II/PAP (according to a Wilcoxon’s bilateral rank sum test with $p=0.05$). The significance test of NSGA-II/PAP against the compared algorithm is summarized in the win-draw-loss (W-D-L) counts.

Problem	NSGA-II/PAP	NSGA-II (Nsize)	NSGA-II (Ngen)	NSGA-II-DE (Nsize)	NSGA-II-DE (Ngen)	NSGA-II/MOE (Nsize)	NSGA-II/MOE (Ngen)
UF1	3.690 × 10${}^{-2}$ ± 2.88 × 10${}^{-4}$	8.938 × 10${}^{-2}$ ± 1.01 × 10${}^{-4}$ †	8.911 × 10${}^{-2}$ ± 7.59 × 10${}^{-4}$ †	7.501 × 10${}^{-2}$ ± 1.80 × 10${}^{-5}$ †	5.993 × 10${}^{-2}$ ± 9.98 × 10${}^{-5}$ †	4.614 × 10${}^{-2}$ ± 7.27 × 10${}^{-5}$ †	2.234 × 10${}^{-2}$ ± 1.99 × 10${}^{-4}$†
UF2	2.180 × 10${}^{-2}$ ± 8.39 × 10${}^{-6}$	3.202 × 10${}^{-2}$ ± 1.68 × 10${}^{-5}$ †	3.086 × 10${}^{-2}$ ± 1.42 × 10${}^{-4}$ †	3.694 × 10${}^{-2}$ ± 4.02 × 10${}^{-5}$ †	2.525 × 10${}^{-2}$ ± 4.63 × 10${}^{-5}$ †	3.771 × 10${}^{-2}$ ± 3.65 × 10${}^{-4}$	1.998 × 10${}^{-2}$ ± 3.84 × 10${}^{-5}$†
UF5	2.677 × 10${}^{-1}$ ± 2.68 × 10${}^{-3}$	2.978 × 10${}^{-1}$ ± 6.01 × 10${}^{-4}$ †	3.481 × 10${}^{-1}$ ± 6.97 × 10${}^{-3}$ †	3.384 × 10${}^{-1}$ ± 2.41× 10${}^{-11}$ †	3.439 × 10${}^{-1}$ ± 7.40 × 10${}^{-4}$ †	3.090 × 10${}^{-1}$ ± 3.29 × 10${}^{-3}$ †	3.772 × 10${}^{-1}$ ± 8.76 × 10${}^{-3}$ †
UF8	1.142 × 10${}^{-1}$ ± 2.93 × 10${}^{-4}$	2.408 × 10${}^{-1}$ ± 4.00 × 10${}^{-4}$ †	1.900 × 10${}^{-1}$ ± 5.01 × 10${}^{-4}$ †	1.959 × 10${}^{-1}$ ± 1.38 × 10${}^{-4}$ †	2.037 × 10${}^{-1}$ ± 6.38 × 10${}^{-5}$ †	1.733 × 10${}^{-1}$ ± 9.14 × 10${}^{-4}$ †	1.530 × 10${}^{-1}$ ± 1.03 × 10${}^{-3}$ †
UF9	6.188 × 10${}^{-2}$ ± 3.61 × 10${}^{-5}$	1.024 × 10${}^{-1}$ ± 1.40 × 10${}^{-3}$ †	1.465 × 10${}^{-1}$ ± 5.17 × 10${}^{-3}$ †	7.863 × 10${}^{-2}$ ± 1.22 × 10${}^{-6}$ †	8.271 × 10${}^{-2}$ ± 2.41 × 10${}^{-6}$ †	2.498 × 10${}^{-1}$ ± 8.84 × 10${}^{-3}$ †	3.536 × 10${}^{-1}$ ± 1.29 × 10${}^{-2}$ †
WFG2	9.415 × 10${}^{-2}$ ± 5.97 × 10${}^{-6}$	1.035 × 10${}^{-1}$ ± 1.84 × 10${}^{-5}$ †	9.365 × 10${}^{-2}$ ± 5.39 × 10${}^{-6}$	1.017 × 10${}^{-1}$ ± 1.36 × 10${}^{-5}$ †	9.319 × 10${}^{-2}$ ± 4.90 × 10${}^{-6}$	9.496 × 10${}^{-2}$ ± 1.20 × 10${}^{-5}$	9.308 × 10${}^{-2}$ ± 8.05 × 10${}^{-6}$
WFG3	8.358 × 10${}^{-2}$ ± 4.31 × 10${}^{-5}$	8.584 × 10${}^{-2}$ ± 2.33 × 10${}^{-5}$	9.104 × 10${}^{-2}$ ± 3.87 × 10${}^{-5}$ †	9.339 × 10${}^{-2}$ ± 5.40 × 10${}^{-5}$ †	1.146 × 10${}^{-1}$ ± 2.33 × 10${}^{-5}$ †	8.391 × 10${}^{-2}$ ± 4.48 × 10${}^{-5}$	8.785 × 10${}^{-2}$ ± 3.88 × 10${}^{-5}$ †
WFG4	1.157 × 10${}^{-1}$ ± 5.75 × 10${}^{-6}$	1.146 × 10${}^{-1}$ ± 2.93 × 10${}^{-6}$	1.242 × 10${}^{-1}$ ± 6.40 × 10${}^{-6}$ †	1.347 × 10${}^{-1}$ ± 5.02 × 10${}^{-6}$ †	1.669 × 10${}^{-1}$ ± 2.03 × 10${}^{-5}$ †	1.156 × 10${}^{-1}$ ± 3.90 × 10${}^{-6}$	1.344 × 10${}^{-1}$ ± 4.64 × 10${}^{-6}$ †
WFG7	1.147 × 10${}^0$ ± 6.21 × 10${}^{-6}$	1.156 × 10${}^0$ ± 6.93 × 10${}^{-6}$ †	1.145 × 10${}^0$ ± 5.47 × 10${}^{-6}$†	1.244 × 10${}^0$ ± 1.55 × 10${}^{-4}$ †	1.176 × 10${}^0$ ± 6.13 × 10${}^{-5}$ †	1.158 × 10${}^0$ ± 6.41 × 10${}^{-6}$ †	1.157 × 10${}^0$ ± 6.12 × 10${}^{-6}$ †
WFG8	2.216 × 10${}^0$ ± 5.05 × 10${}^{-5}$	2.182 × 10${}^0$ ± 5.23 × 10${}^{-6}$ †	2.149 × 10${}^0$ ± 5.86 × 10${}^{-7}$†	2.796 × 10${}^0$ ± 1.03 × 10${}^{-3}$ †	2.475 × 10${}^0$ ± 1.80 × 10${}^{-3}$ †	2.204 × 10${}^0$ ± 5.52 × 10${}^{-6}$ †	2.159 × 10${}^0$ ± 3.14 × 10${}^{-5}$ †
DTLZ1	1.083 × 10${}^{-2}$ ± 6.97 × 10${}^{-5}$	3.900 × 10${}^{-3}$ ± 1.69 × 10${}^{-7}$ †	2.250 × 10${}^{-3}$ ± 5.65 × 10${}^{-9}$†	4.432 × 10 ± 6.57 × 10${}^{-1}$ †	2.194 × 10${}^{-3}$ ± 5.87 × 10${}^{-9}$ †	1.581 × 10${}^{-2}$ ± 3.93 × 10${}^{-3}$ †	8.446 × 10${}^{-2}$ ± 3.04 × 10${}^{-2}$ †
DTLZ6	5.596 × 10${}^{-3}$ ± 9.35 × 10${}^{-8}$	8.356 × 10${}^{-3}$ ± 4.43 × 10${}^{-7}$ †	5.887 × 10${}^{-3}$ ± 4.79 × 10${}^{-8}$ †	8.817 × 10${}^{-3}$ ± 1.28 × 10${}^{-6}$ †	5.767 × 10${}^{-3}$ ± 5.98 × 10${}^{-8}$ †	1.019 × 10${}^{-2}$ ± 2.26 × 10${}^{-6}$ †	5.208 × 10${}^{-3}$ ± 4.25 × 10${}^{-8}$†
DTLZ7	5.105 × 10${}^{-3}$ ± 2.05 × 10${}^{-8}$	8.745 × 10${}^{-3}$ ± 7.24 × 10${}^{-7}$ †	2.097 × 10${}^{-2}$ ± 6.83 × 10${}^{-3}$ †	8.962 × 10${}^{-3}$ ± 1.33 × 10${}^{-6}$ †	6.744 × 10${}^{-3}$ ± 6.79 × 10${}^{-8}$ †	9.539 × 10${}^{-3}$ ± 1.39 × 10${}^{-6}$ †	3.609 × 10${}^{-2}$ ± 1.32 × 10${}^{-2}$ †
ZDT1	4.528 × 10${}^{-3}$ ± 3.32 × 10${}^{-8}$	7.800 × 10${}^{-3}$ ± 1.10 × 10${}^{-6}$ †	4.660 × 10${}^{-3}$ ± 3.30 × 10${}^{-8}$ †	7.732 × 10${}^{-3}$ ± 1.08 × 10${}^{-6}$ †	4.593 × 10${}^{-3}$ ± 2.87 × 10${}^{-8}$	8.253 × 10${}^{-3}$ ± 1.52 × 10${}^{-6}$ †	5.014 × 10${}^{-3}$ ± 1.16 × 10${}^{-7}$ †
ZDT2	4.748 × 10${}^{-3}$ ± 5.31 × 10${}^{-8}$	7.711 × 10${}^{-3}$ ± 4.20 × 10${}^{-7}$ †	5.386 × 10${}^{-3}$ ± 6.72 × 10${}^{-8}$ †	7.955 × 10${}^{-3}$ ± 7.53 × 10${}^{-7}$ †	4.668 × 10${}^{-3}$ ± 3.52 × 10${}^{-8}$	9.236 × 10${}^{-3}$ ± 6.66 × 10${}^{-6}$ †	5.077 × 10${}^{-3}$ ± 1.26 × 10${}^{-7}$ †
W-D-L	-	11-2-2	11-1-3	15-0-0	12-3-0	11-4-0	10-1-4

and

Table 8. The mean ± standard deviation of the IGD values across the 30 runs over the testing set. For each testing problem, the best performance is indicated in bold (note for IGD, the smaller, the better). † indicates that the performance of the algorithm is significantly different from the performance of NSGA-II/PAP (according to a Wilcoxon’s bilateral rank sum test with $p=0.05$). The significance test of NSGA-II/PAP against the compared algorithm is summarized in the win-draw-loss (W-D-L) counts.

Problem	NSGA-II/PAP	MOEA/D (Ngen)	MOPSO (Ngen)	MOEA/D (Nsize)	MOPSO (Nsize)
UF1	3.690 × 10${}^{-2}$ ± 2.88 × 10${}^{-4}$	3.789 × 10${}^{-2}$ ± 2.31 × 10${}^{-4}$ †	3.584 × 10${}^{-2}$ ± 2.16 × 10${}^{-4}$ †	2.620 × 10${}^{-2}$ ± 1.63 × 10${}^{-4}$†	3.087 × 10${}^{-2}$ ± 2.47 × 10${}^{-4}$ †
UF2	2.180 × 10${}^{-2}$ ± 8.39 × 10${}^{-6}$	2.754 × 10${}^{-2}$ ± 3.01 × 10${}^{-4}$ †	2.790 × 10${}^{-2}$ ± 1.92 × 10${}^{-6}$ †	2.185 × 10${}^{-2}$ ± 2.01 × 10${}^{-4}$	2.088 × 10${}^{-2}$ ± 6.01 × 10${}^{-7}$†
UF5	2.677 × 10${}^{-1}$ ± 2.68 × 10${}^{-3}$	4.470 × 10${}^{-1}$ ± 2.36 × 10${}^{-2}$ †	5.405 × 10${}^{-1}$ ± 3.96 × 10${}^{-2}$ †	2.821 × 10${}^{-1}$ ± 4.72 × 10${}^{-3}$ †	6.174 × 10${}^{-1}$ ± 7.04 × 10${}^{-2}$ †
UF8	1.142 × 10${}^{-1}$ ± 2.93 × 10${}^{-4}$	1.142 × 10${}^{-1}$ ± 7.01 × 10${}^{-4}$	1.149 × 10${}^{-1}$ ± 9.64 × 10${}^{-4}$ †	1.154 × 10${}^{-1}$ ± 3.17 × 10${}^{-4}$ †	1.147 × 10${}^{-1}$ ± 3.63 × 10${}^{-4}$ †
UF9	6.188 × 10${}^{-2}$ ± 3.61 × 10${}^{-5}$	7.184 × 10${}^{-2}$ ± 3.67 × 10${}^{-3}$ †	2.070 × 10${}^{-1}$ ± 9.66 × 10${}^{-3}$ †	6.776 × 10${}^{-2}$ ± 1.08 × 10${}^{-4}$ †	8.757 × 10${}^{-2}$ ± 3.96 × 10${}^{-3}$ †
WFG2	9.415 × 10${}^{-2}$ ± 5.97 × 10${}^{-6}$	1.281 × 10${}^{-1}$ ± 3.42 × 10${}^{-6}$ †	9.076 × 10${}^{-2}$ ± 2.55 × 10${}^{-6}$ †	8.935 × 10${}^{-2}$ ± 4.83 × 10${}^{-7}$ †	3.913 × 10${}^{-2}$ ± 1.61 × 10${}^{-7}$†
WFG3	8.358 × 10${}^{-2}$ ± 4.31 × 10${}^{-5}$	1.283 × 10${}^{-1}$ ± 6.10 × 10${}^{-5}$ †	1.544 × 10${}^{-1}$ ± 6.11 × 10${}^{-5}$ †	8.353 × 10${}^{-2}$ ± 1.28 × 10${}^{-5}$	8.589 × 10${}^{-2}$ ± 4.04 × 10${}^{-6}$ †
WFG4	1.157 × 10${}^{-1}$ ± 5.75 × 10${}^{-6}$	1.235 × 10${}^{-1}$ ± 3.99 × 10${}^{-6}$ †	2.104 × 10${}^{-1}$ ± 5.92 × 10${}^{-5}$ †	9.481 × 10${}^{-2}$ ± 3.81 × 10${}^{-6}$†	1.533 × 10${}^{-1}$ ± 7.97 × 10${}^{-7}$ †
WFG7	1.147 × 10${}^0$ ± 6.21 × 10${}^{-6}$	1.278 × 10${}^0$ ± 2.43 × 10${}^{-5}$ †	1.290 × 10${}^0$ ± 8.17 × 10${}^{-5}$ †	1.267 × 10${}^0$ ± 8.03 × 10${}^{-6}$ †	1.264 × 10${}^0$ ± 9.01 × 10${}^{-5}$ †
WFG8	2.216 × 10${}^0$ ± 5.05 × 10${}^{-5}$	2.175 × 10${}^0$ ± 2.79 × 10${}^{-6}$ †	2.912 × 10${}^0$ ± 4.51 × 10${}^{-5}$ †	2.173 × 10${}^0$ ± 9.51 × 10${}^{-7}$†	2.889 × 10${}^0$ ± 1.60 × 10${}^{-5}$ †
DTLZ1	1.083 × 10${}^{-2}$ ± 6.97 × 10${}^{-5}$	3.176 × 10${}^{-2}$ ± 7.46 × 10${}^{-8}$ †	2.728 × 10${}^1$ ± 2.44 × 10${}^2$ †	5.318 × 10${}^{-3}$ ± 2.49 × 10${}^{-10}$†	3.789 × 10${}^1$ ± 2.85 × 10${}^2$ †
DTLZ6	5.596 × 10${}^{-3}$ ± 9.35 × 10${}^{-8}$	7.114 × 10${}^{-3}$ ± 3.14 × 10${}^{-7}$ †	6.585 × 10${}^{-3}$ ± 1.76 × 10${}^{-7}$ †	6.174 × 10${}^{-3}$ ± 1.62 × 10${}^{-9}$ †	5.681 × 10${}^{-3}$ ± 1.52 × 10${}^{-8}$
DTLZ7	5.105 × 10${}^{-3}$ ± 2.05 × 10${}^{-8}$	1.062 × 10${}^{-2}$ ± 2.15 × 10${}^{-6}$ †	7.098 × 10${}^{-3}$ ± 1.92 × 10${}^{-7}$ †	5.581 × 10${}^{-3}$ ± 8.04 × 10${}^{-9}$ †	5.640 × 10${}^{-3}$ ± 1.39 × 10${}^{-8}$ †
ZDT1	4.528 × 10${}^{-3}$ ± 3.32 × 10${}^{-8}$	6.981 × 10${}^{-3}$ ± 4.99 × 10${}^{-7}$ †	6.856 × 10${}^{-3}$ ± 1.61 × 10${}^{-7}$ †	4.655 × 10${}^{-3}$ ± 4.97 × 10${}^{-9}$ †	5.371 × 10${}^{-3}$ ± 2.10 × 10${}^{-9}$ †
ZDT2	4.748 × 10${}^{-3}$ ± 5.31 × 10${}^{-8}$	6.953 × 10${}^{-3}$ ± 2.93 × 10${}^{-7}$ †	5.922 × 10${}^{-3}$ ± 7.65 × 10${}^{-8}$ †	5.121 × 10${}^{-3}$ ± 2.46 × 10${}^{-9}$ †	4.398 × 10${}^{-3}$ ± 3.50 × 10${}^{-9}$ †
W-D-L	-	13-1-1	13-0-2	8-2-5	11-1-3

. For each problem, the best performance is indicated in bold, and a Wilcoxon’s bilateral rank-sum test was conducted to judge whether the difference between the performance of the compared algorithm and the performance of NSGA-II/PAP was significant (with p = 0.05).

In summary, NSGA-II/PAP not only shows a superior performance to conventional single-operator-based MOEAs such as NSGA-II, but also outperforms the human-designed multi-operator-based NSGA-II/MOE. Moreover, the performance advantage of NSGA-II is still significant even when the compared algorithms consume the same number of FEs (indicated by Ngen and Nsize as NSGA-II/PAP. We further compared the average runtime of NSGA-II/PAP and the two variants of NSGA-II/MOE for solving the testing problems. In addition, we also compared the running time of MOEAs/PAP in parallel and serial cases. The results in terms of speed-up ratios are illustrated in Figure 2. It can be observed that the speed-up ratios achieved by running NSGA-II/PAP in parallel are worse than the ideal ratios, which are equal to the number of member algorithms. The reason for this is that the current naive parallel-running strategy in NSGA-II/PAP only runs member algorithms in parallel and does not parallelize the Restructure procedure. We note that parallelizing the Restructure procedure can further improve the speed-up ratio, which is an important direction for future research.

Overall, the above results indicate that PAPs are, in general, a powerful tool for solving computationally hard problems, and the automatic construction of PAPs has huge potential in developing powerful MOP solvers that can even rival human-designed state-of-the-art MOEAs.

5.5. Performances of Member Algorithms

Table 9. The mean of the IHVR values achieved by the member algorithms of NSGA-II/PAP, the variant of NSGA-II/PAP without the Restructure procedure, and NSGA-II/PAP, across 30 independent runs over the training set. For each problem, the best performance is indicated in bold (note that, for IHVR, the larger, the better), and the best performance achieved among the member algorithms is indicated by underline.

Problem	Member Algorithm 1	Member Algorithm 2	Member Algorithm 3	Member Algorithm 4	Member Algorithm 5	Member Algorithm 6	No Restructure	NSGA-II/PAP
UF3	0.575877	0.384182	0.514065	0.427070	0.545803	0.577853	0.597666	0.630382
UF4	0.981580	0.986358	0.984794	0.982849	0.981665	0.981615	0.986358	0.986358
UF6	0.544830	0.378446	0.409731	0.603205	0.472694	0.504911	0.648320	0.791040
UF7	0.805299	0.821830	0.789846	0.884180	0.768611	0.791250	0.890959	0.931463
UF10	0.124504	0.442486	0.544413	0.171657	0.240853	0.113756	0.621673	0.621673
WFG1	0.549970	0.116983	0.417594	0.136134	0.732619	0.500422	0.748021	0.757083
WFG5	0.845592	0.837363	0.838631	0.841177	0.850955	0.843478	0.850955	0.852089
WFG6	0.827467	0.869466	0.821787	0.890711	0.826054	0.813707	0.893355	0.893355
WFG9	0.704421	0.582577	0.684037	0.662125	0.707536	0.705735	0.707590	0.707642
DTLZ2	0.993334	0.994437	0.993827	0.994103	0.993781	0.993154	0.994510	0.994510
DTLZ3	0.933205	0.000739	0.010216	0.001359	0.006734	0.785523	0.933205	0.944154
DTLZ4	0.993408	0.994484	0.973001	0.994136	0.993670	0.993355	0.994495	0.994495
DTLZ5	0.993334	0.994437	0.993826	0.994103	0.993781	0.993153	0.994510	0.994510
ZDT3	0.993333	0.994189	0.888358	0.996239	0.975032	0.993530	0.996299	0.996299
ZDT4	0.961404	0.115557	0.273397	0.062209	0.702779	0.965560	0.971306	0.972525
ZDT5	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000
ZDT6	0.989010	0.990608	0.312566	0.992934	0.971480	0.990346	0.993043	0.993043

reports the independent performance of each member algorithm of NSGA-II/PAP, and the performance of NSGA-II/PAP, on the training problems in terms of the metric IHVR (see Section 4.2). The details of these six member algorithms can be found in Table 3. As before, on each problem, the best performance is indicated in bold; moreover, the best performance achieved among the member algorithms is indicated by an underline. One can observe that the performance of different member algorithms vary a lot. For example, member algorithm 1 performed poorly on UF10 but achieved an excellent performance on DTLZ3, while member algorithm 3 performed exactly the opposite. Actually, noticing that every member algorithm column in Table 9 has at least one cell underlined, this means each member algorithm achieved the best performance among all the member algorithms on at least one problem. Leveraging such performance complementarity eventually leads to a powerful PAP, i.e., NSGA-II/PAP, which performed better than any single member algorithm on all the problems.

5.6. Effectiveness of the Restructure Procedure

To investigate the effect of the Restructure procedure, we removed it from NSGA-II/PAP and tested the resulting PAP on the training set. Note that this PAP is a conventional PAP as defined in Equation (11), where the best solution set found by the member algorithms would be returned as its output. The “No Restructure” column in Table 9 presents the results. It can be observed that, due to the integration of the Restructure procedure, NSGA-II/PAP achieved a performance improvement on eight out of seventeen problems, clearly verifying the effectiveness of the procedure. Figure 3 further demonstrates that, in the construction process, as more member algorithms are included in the PAP, the Restructure procedure could bring a larger performance improvement.

6. Conclusions

This work extended the realm of the automatic construction of PAPs to multi-objective optimization. Specifically, we proposed a variant of PAP, namely MOEAs/PAP, which involves a Restructure procedure to better determine the output solution set of the PAP. Then, we presented an automatic construction approach for MOEAs/PAP, which uses a novel metric to evaluate the performance of MOEAs/PAPs across multiple MOPs. Based on a training set of MOPs and an algorithm configuration space defined by several variants of NSGA-II, our approach could build MOEAs/PAPs that outperformed existing single-operator based-MOEAs and the state-of-the-art multi-operator-based MOEAs designed by human experts.

The promising results presented in this work have indicated the huge potential of the automatic construction of PAPs in the area of multi-objective optimization. Further directions for investigations are listed below.

• The algorithm configuration space used in this work is still defined based on the general algorithm framework of NSGA-II. In the literature, there have been some studies on developing highly parameterized MOEA frameworks [45,46]. It is valuable to apply our construction approach to these MOEA frameworks, hopefully leading to even better MOEAs/PAPs.
• When constructing MOEAs/PAPs, it is important to maintain the diversity among the member algorithms. Hence, the population diversity preservation schemes, such as negatively correlated search [47], can be introduced into the construction approach to promote cooperation between different member algorithms.
• In real-world applications, one may be unable to collect sufficient MOPs as training problems. How to automatically build powerful PAPs in these scenarios is also worth studying.
• The effectiveness of MOEAs/PAP has been primarily demonstrated through experimental evidence, but with an absence of theoretical analysis. A more thorough investigation of its exceptional performance is crucial for advancing our understanding, which, in turn, can lead to enhancements in its design and the development of a more comprehensive automatic construction algorithm.