Cluster-Based Regression Transfer Learning for Dynamic Multi-Objective Optimization

Abstract

Many multi-objective optimization problems in the real world have conflicting objectives, and these objectives change over time, known as dynamic multi-objective optimization problems (DMOPs). In recent years, transfer learning has attracted growing attention to solve DMOPs, since it is capable of leveraging historical information to guide the evolutionary search. However, there is still much room for improvement in the transfer effect and the computational efficiency. In this paper, we propose a cluster-based regression transfer learning-based dynamic multi-objective evolutionary algorithm named CRTL-DMOEA. It consists of two components, which are the cluster-based selection and cluster-based regression transfer. In particular, once a change occurs, we employ a cluster-based selection mechanism to partition the previous Pareto optimal solutions and find the clustering centroids, which are then fed into autoregression prediction model. Afterwards, to improve the prediction accuracy, we build a strong regression transfer model based on TrAdaboost.R2 by taking advantage of the clustering centroids. Finally, a high-quality initial population for the new environment is predicted with the regression transfer model. Through a comparison with some chosen state-of-the-art algorithms, the experimental results demonstrate that the proposed CRTL-DMOEA is capable of improving the performance of dynamic optimization on different test problems.

1. Introduction

In the real world, many multi-objective optimization problems [1] have multiple conflicting objectives that may change over time. Such problems are called dynamic multi-objective optimization problems (DMOPs) [2]. In recent years, the research on solving DMOPs has attracted more and more researchers and there have been lots of optimization methods developed [3,4,5]. Multi-objective evolutionary algorithms (MOEAs) have been widely applied to solve DMOPs in various areas, such as wireless sensor networks [6], financial optimization problems [7], path planning [8] and so on. When applied to solve DMOPs, traditional MOEAs [9,10,11,12] should be improved to adapt to the dynamisms, which are capable of tracking the changing Pareto optimal fronts (POFs) and providing a diverse set of Pareto optimal solutions (POSs) over time.

To solve DMOPs, there are various kinds of dynamic MOEAs (DMOEAs) in the literature, which can be categorized as follows: diversity approaches [13,14,15], memory mechanisms [16,17,18], and prediction-based methods [19,20,21]. Generally, the diversity approaches include increasing diversity [22], maintaining diversity [15], and multi-population strategy [23]. More specifically, the environmental adaption of population diversity can be addressed with increasing diversity by adding variety to the population after the detection of a change, maintaining diversity by avoiding population convergence to track the time-varying POS throughout the run, or dividing the population into some different subpopulations. Additionally, a variety of memory mechanisms are designed to store historical information in the past environment and reuse these information so as to save computational costs or guide the future search direction.

Among various DMOEAs, the prediction-based methods take advantage of the previous search information to predict future POSs and have drawn lots of attention recently. [24] proposed a feed-forward prediction strategy to estimate the new POS, aiming at improving the convergence speed to the new POF. However, this strategy ignores the distribution characteristics of POF and affects the prediction efficiency. Zhou et al. [25] put forward a novel prediction-based population re-initialization method to predict the new locations of optimal solutions when a change occurs.

In recent years, transfer learning [26,27] has been considered to be capable of effectively improving the prediction performance. For DMOPs, the dynamic nature at two adjacent time steps may share certain common features, and thereby the solutions obtained from the previous environment can provide useful knowledge for the new individuals during the optimization process. Jiang et al. [28] proposed a DMOEA based on transfer learning, named Tr-DMOEA, to predict an initial population by learning from the previous evolutionary process. In their work, transfer component analysis (TCA) [29] is applied to find a latent space where the objective values of solutions in the target domain are close to that of solutions in the source domain. Besides, to improve the computational efficiency, several transfer learning-based DMOEAs have been presented in [30,31,32], where the promising solutions in the new environment are predicted with the historical information of past environments using individual-based methods [32], manifold transfer learning [31], knee point-based imbalanced transfer learning [30], etc.

Even though remarkable progress has been made in transfer learning-based DMOEAs, there is still much room for improvement in the transfer performance. First, as mentioned in [32], transferring a large number of common solutions consumes a large amount of computational resources, and the negative transfer can easily occur due to solution aggregation. Second, most of the existing algorithms transfer knowledge through a latent space that requires more parameters and takes excessive computing time.

In view of the above shortcomings, we propose a cluster-based regression transfer learning method-based DMOEA, called CRTL-DMOEA, which consists of two stages, i.e., cluster-based selection and cluster-based regression transfer. Specifically, once a change occurs, the cluster-based selection mechanism is first employed to find the centroids of approximate POSs by clustering the previous POSs with localPCA [33], which are then fed into autoregression (AR) [34] model. Afterwards, to improve the prediction accuracy, we build an regression transfer model based on TrAdaboost.R2 [35] by taking advantage of the knowledge from the clustering centroids. Finally, a high-quality initial population is predicted with the assistance of the regression transfer model for the new environment.

The main contributions of this paper are given as follows:

• In this paper, we present a cluster-based selection mechanism by clustering the previous POSs with localPCA, and predicting the centroids in each cluster with AR model. Selecting the representative individuals to transfer can save a lot of computational time and the effect of transfer.
• This paper proposes a cluster-based regression transfer method based on the TrAdaboost.R2 to leverage the information from clustered centroids in historical environment. The method constructs a regression transfer model, which does not need setting more hyperparameters and improve the computational complexity.
• The proposed algorithm has been shown to be effective by comparing with other state-of-the-art methods on different types of benchmark problems.

Section 2 introduces the background and related work. Section 3 elaborates on CRTL-DMOEA in detail. Section 4 presents the experimental design and results. Section 5 concludes and discusses future research.

2. Background and Related Work

2.1. Dynamic Multi-Objective Optimization

Without the loss of generality, we consider the minimization problem and the DMOP is mathematically defined as follows:

$$\begin{array}{ll}min& F(x,t)=\{f_1(x,t),f_2(x,t),\cdots ,f_M(x,t)\}\hfill \\ \mathrm{s}.\mathrm{t}.& x\in \mathsf{\Omega };\hfill \end{array}$$

(1)

where $x=(x_1,x_2,\cdots ,x_n)$ is the n-dimension decision variable bounded in the decision space $\mathsf{\Omega }$. t represents the environment variable. $F(x,t)$ denotes the M-dimensional objective vector.

Definition 1.

(Dynamic Pareto Dominance): At time t, $x_1$ is said to Pareto dominate $x_2$, denoted by $x_1{\prec }_t{x}_2$, if and only if

$$\left\{\begin{array}{cc}\forall i=1,\cdots ,M,\hfill & f_i(x_1,t)\le f_i(x_2,t)\hfill \\ \exists i=1,\cdots ,M,\hfill & f_i(x_1,t)<f_i(x_2,t)\hfill \end{array}\right.$$

(2)

Definition 2.

(Dynamic Pareto Optimal Set (DPOS)): If a solution $x^{*}$ is not dominated by any other solution, $x^{*}$ is called a Pareto optimal solution. All $x^{*}$ at time t form the DPOS, denoted by

$$\mathit{DPOS}=\{x^{*}\mid \nexists x^{*}\in \mathsf{\Omega },x{\prec }_t{x}^{*}\}$$

(3)

Definition 3.

(Dynamic Pareto Optimal Front (DPOF)): $F(x,t)$ is the objective function with respect to time t. DPOF is defined as follows:

$$\mathit{DPOF}=\{x^{*}\mid x^{*}\in DPOS\}$$

(4)

2.2. Related Work

At present, the key components of DMOEAs are environmental change detection, change response strategy and static multi-objective EA (MOEA). The environmental change detection is mainly used to detect whether change occurs in the environment. The state-of-the-art research mainly focuses on three aspects, including re-evaluation [3,22,34,36], distribution estimation of objective value [37], and steady-state detection [38]. In general, the most common detection mechanism is performed to re-evaluate the best solution, or some other solutions as detectors. If the objective values are different in adjacent times, we judge that the change has been detected. Jiang et al. [38] proposed a steady-state change detection method based on re-evaluation. Instead of selecting a proportion of population members as sentinels, they check the whole population in random order one by one. Afterwards, a change is assumed to be detected if a discrepancy is found in one member and there is no need to do further evaluation.

In the literature, various change response strategies have been proposed to track the POS of the new environment quickly by initializing the population and respond to the changed environment in time, which are the core component of DMOEAs. Generally, they can be mainly classified as follows: diversity approaches [13,14,15], memory mechanisms [16,17,18], and prediction-based methods [19,20,21].

The diversity approaches handle DMOPs by increasing diversity [22] and maintaining diversity [15], as well as through multi-population strategies [23]. The increasing diversity methods generally take some explicit actions such as reinitialization or hypermutation when a change occurs. Jiang et al. [38] proposed a change respond mechanism to maintain a balanced level of population diversity and convergence. The increasing diversity methods blindly respond to the changing environment, probably resulting in misleading the optimization process. Most of the maintaining diversity methods tend to keep a certain level of diversity and thereby adapt more easily to changes and explore the new search space. Grefenstette [13] proposed a random immigrant generic algorithm to replace some individuals by randomly generated ones in every generation. Multi-population approach is to maintain multiple sub-populations at the same time and do the exploration or exploitation tasks separately. This kind of approaches is known to be effective for solving multiple peaks or the competing peaks problems. Wang et al. [39] proposed that multiple sub-populations can be generated adaptively based on a set of single-objective sub-problems decomposed from a MOP.

As for the memory mechanisms, Yang [40] proposed an associative memory scheme for genetic algorithms, in which both the optimal individuals and the environmental information are stored in the memory and leveraged to generate a new population when a change has been recognized. Goh and Tan [3] proposed a competitive–cooperative co-evolutionary algorithm for DMOPs, in which a temporary memory method is used to store the previous solutions in the archive.

Significantly, the prediction-based methods have shown to be effective to reuse the historical information to predict the future individuals in handling DMOPs. Koo et al. [41] proposed a gradient strategy for DMOPs which predicts the direction and magnitude of the next change based on the historical solutions. However, such methods assume that the training and test data should have the same distribution, which may not become true in many real-world DMOPs. Integrating transfer learning into DMOEAs [42,43] is effective to address this issue, which could improve the learning performance by avoiding much expensive efforts in data labeling. However, to further improve the computationally intensive property and overcome the negative transfer remain great challenges when dealing with DMOPs.

3. Proposed Methods

In this section, we propose a cluster-based regression transfer learning method-based DMOEA, called CRTL-DMOEA, to handle DMOPs. Two main components, i.e., cluster-based selection and cluster-based regression transfer, as shown in Figure 1, are unified into one framework to generate an excellent initial population to help the MOEA find the changing POS efficiently and effectively.

The schematic of the proposed CRTL-DMOEA is provided in Figure 2. Briefly, CRTL-DMOEA randomly initializes the population. If an environmental change is detected, the cluster-based selection mechanism is first employed to find the centroids of approximate POSs and estimate the centroids under new environment with AR model. Subsequently, an regression transfer model based on TrAdaboost.R2 [35] is constructed to transfer the knowledge from the estimated centroids and obtain a high-quality initial population for the new environment. Finally, the new generated initial population are optimized by static MOEAs to converge towards POSs at different environments.

3.1. The Overall Framework

The pseudo-code of CRTL-DMOEA is given in Algorithm 1. In the following, we will describe the overall framework in detail.

• Line 1-2: The initial population $initPop$ is randomly generated in the decision space and optimized by MOEA.
• Line 3: Check for environmental changes.
• Line 4: The time t is increased by 1, if the environment changes.
• Line 5: The clustering centroids C and corresponding estimated centroids $C_{es}$ are found by Cluster-Based Selection, which will be described in Section 3.2.
• Line 6: Two sets of population $P_t$ and $P_{t-1}$ are randomly generated.
• Line 7-8: The clustering centroids and $P_{t-1}$, together with their objective values at time $t-1$ are regarded as the source domain $D_{so}$.
• Line 9-10: The estimated centroids $C_{es}$ and $P_t$ are merged with their current objective values to serve as the target domain $D_{ta}$.
• Line 11: Cluster-Based Regression Transfer utilizes $D_{so}$ and $D_{ta}$ to generate the initial population $initPop$, which will be described in Section 3.3.
• Line 12: $initPop$ will be further optimized by MOEA.

Algorithm 1 The framework of CRTL-DMOEA.

Input:$F(x,t)$: the dynamic optimization problem; MOEA: a static MOEA; $N_c$: the number of clusters.

Output: The POS of $F(x,t)$ in different environments.

1:  Randomly initialize N individuals $initPop$;

2:  $PO{S}_0$ = MOEA$(initPop,F_O\left(x\right))$;

3:  while change detected do

4:       $t=t+1$;

5:       $(C,C_{es})$ = Cluster-Based Selection$(F(x,t),PO{S}_{t-1},N_c)$;

6:       Generate randomly two sets of solutions $P_t$ and $P_{t-1}$;

7:       $X_{so}=C\cup P_{t-1}$;

8:       $D_{so}=X_{so}\cup F(X_{so},t-1)$;

9:       $X_{ta}=C_{es}\cup P_t$;

10:       $D_{ta}=X_{ta}\cup F(X_{ta},t)$;

11:       $initPop$ = Cluster-Based Regression Transfer$(D_{so},D_{ta},N_c)$;

12:       $PO{S}_t$ = MOEA$(initPop,F(x,t\left)\right)$;

13:  end while

3.2. Cluster-Based Selection

The goal of cluster-based selection is to generate the estimated centroids, which are used to construct the transfer model in the next step. We use localPCA [33] to cluster the previous POS and find the centroids with excellent convergence and diversity, which are then predicted by AR [34] model to obtain the estimated centroids.

To depict the procedure of cluster-based selection, the pseudo-code is given in Algorithm 2. Firstly, we set the clustering centroids and estimated centroids as empty set in line 1. Then, localPCA was applied to partition the previous approximate $PO{S}_{t-1}$ into $N_c$ subpopulation in line 2. Specifically, we partitioned the individuals in $PO{S}_{t-1}$ into disjoint $N_c$ clusters according to the distances from the individual to the principal subspace of the points in each cluster. Afterwards, in line 4, the centroid $c_i$ of cluster $C_i$ can be obtained by

$$c_i=\frac{{\displaystyle \sum _{x_k\in C_i}}{x}_k}{|C_i|}$$

(5)

where $x_k$ stands for the kth individual in cluster $C_i$ and $|·|$ means the cardinality. All the $c_i$ in each cluster form the set of clustering centroids C in line 5. Subsequently, the AR model was constructed for prediction in line 6, with more details introduced in [34]. Finally, we obtained the estimated centroids $C_{es}$ in line 7, which were subsequently used to train the regression transfer model for the new environment.

Algorithm 2 Cluster-Based Selection.

Input:$F(x,t)$: the dynamic optimization problem; $PO{S}_{t-1}$: the POS at time $t-1$; $N_c$: the number of clusters.

Output: The set of clustering centroids and estimated centriods C and $C_{es}$.

1:  Set $C=⌀$ and $C_{es}=⌀$;

2:  Clustering $PO{S}_{t-1}$ into $C_1$, $C_2,\cdots$, $C_{N_c}$ by localPCA;

3:  for $i=1:N_c$ do

4:       Calculate the centroid $c_i$ of each cluster by Equation (5);

5:       $C=C\cup c_i$;

6:       Predict the clustering centroids by AR model and obtain estimated centroid $e{s}_i$;

7:       $C_{es}=C_{es}\cup e{s}_i$;

8:  end for

9:  return$(C,C_{es})$;

3.3. Cluster-Based Regression Transfer Method

In this section, we propose a cluster-based regression transfer method based on TrAdaboost.R2. Our motivation is to save the computational cost while improving the transfer effect. This method constructs a strong regression model $h_s^t$ at time t, which can be used to filter out a high-quality initial population for the next time moment.

In the following, we will describe the details of this method as shown in Algorithm 3. The source domain $D_{so}$ consists of the clustering centroids, $P_{t-1}$, and their objective values at time $t-1$. The target domain $D_{ta}$ includes the estimated centroids $C_{es}$, $P_t$ and their objective values at time t.

• Line 1: The initial population $initPop$ is an empty set.
• Line 2: $D_{so}$ and $D_{ta}$ are merged into one set D.
• Line 3-4: Initialize the weight $w_1\left(x\right)=1/\left|D\right|$ for all individuals and set the number of iterations $K_{max}$.
• Line 5-10: $K_{max}$ weak regression models are trained with TrAdaboost.R2.
• Line 11: The strong regression model $h_s^t$ is constructed by combining the final $\lceil \frac{K_{max}}{2}\rceil$ weak models.
• Line 12-13: We randomly sample a large number of test samples $X_{test}$, which are predicted by the regression model $h_s^t$ to get the predicted objective values $h_s^t\left(X_{test}\right)$.
• Line 14: $X_{test}$ are ranked by non-dominated sorting based on the estimated objective values, and the non-dominated solutions are stored as $initPop$.
• Line 15-20: If the size of $initPop$ exceeds N, we randomly select some solutions to truncate; otherwise, some Gaussian noises will be added.
• Line 21: $initPop$ serves as the initial population for the static MOEA to be optimized in the new environment.

More specifically, when constructing the weak regression models with TrAdaboost.R2, the weight $w_1\left(x\right)$ is firstly initialized as $1/\left|D\right|$ for each individual. Then, to train each weak regression models $h_i^t$, we call a base learner Support Vector Regression (SVR) [44] with D and $w_i^t$. The error between the true objective value $F(x,t)$ and the weak regression model $h_i^t$ are mapped into an adjusted error $e_i^t\left(x\right)$, which is expressed as:

$$e_i^t\left(x\right)=\left\{\begin{array}{cc}\frac{|F(x,t-1)-h_i^t\left(x\right)|}{D_i^t},\hfill & x\in D_{so}\hfill \\ \frac{|F(x,t)-h_i^t\left(x\right)|}{D_i^t},\hfill & x\in D_{ta}\hfill \end{array}\right.$$

(6)

where

$$\begin{array}{cc}\hfill D_i^t=& max\left\{\underset{x\in D_{so}}{max}\right|F(x,t-1)-h_i^t\left(x\right)|,\hfill \\ \hfill & \underset{x\in D_{ta}}{max}|F(x,t)-h_i^t\left(x\right)\left|\right\}\hfill \end{array}$$

(7)

After that, we calculate the adjusted error for $h_i^t$ by

$${ϵ}_i^t=\sum _{x\in D_{ta}}{e}_i^t\left(x\right)w_i^t\left(x\right)$$

(8)

Then, the weight for each individual $w_i^t\left(x\right)$ is updated based on the adjusted error $e_i^t\left(x\right)$ and ${ϵ}_i^t$. We treat the training data in D differently, which means that, if an individual have a large adjusting error $e_i^t\left(x\right)$, we increase its weight $w_i^t\left(x\right)$ if it belongs to $D_{ta}$ and decrease its weight if it is from $D_{so}$. The update of weights can be calculated as

$$w_{i+1}^t\left(x\right)=\left\{\begin{array}{cc}{w}_i^t\left(x\right){\beta }^{e_i^t\left(x\right)},\hfill & x\in D_{so}\hfill \\ w_i^t\left(x\right){\beta }_i^{-e_i^t\left(x\right)},\hfill & x\in D_{ta}\hfill \end{array}\right.$$

(9)

where

$$\begin{array}{cc}\hfill {\beta }_i& =\frac{{ϵ}_i^t}{1-{ϵ}_i^t}\hfill \\ \hfill \beta & =\frac{1}{1+\sqrt{2In|D_{so}|/K_{max}}}\hfill \end{array}$$

(10)

Following this, the individuals with large weights are adapted to the target domain, which is helpful for the base learner to train subsequent regression models.

Algorithm 3 Cluster-Based Regression Transfer.

Input:$D_{so}$: the source data set; $D_{ta}$: the target data set; $N_c$: the number of clusters; N: the population size.

Output: The initial population $initPop$.

1:  Set $initPop=⌀$;

2:  $D=D_{so}\cup D_{ta}$;

3:  Initialize the weight $w_1\left(x\right)=1/\left|D\right|$ for all individuals;

4:  Set the number of iterations $K_{max}$;

5:  for $i=1$ to $K_{max}$ do

6:       Use SVR to train a weak regression model $h_i^t$ with D and $w_i^t$;

7:       Compute the adjusted error $e_i^t\left(x\right)$ by Equation (6) for each individual;

8:       Compute the adjusted error of $h_i^t$ by Equation (8);

9:       Update the weight $w_{i+1}^t\left(x\right)$ by Equation (9);

10:  end for

11:  Obtain the strong regression model $h_s^t$ by combining the final $\lceil \frac{K_{max}}{2}\rceil$ weak models;

12:  Randomly generate a large number of test solutions $X_{test}$;

13:  Apply $h_s^t$ to predict the objective value $h_s^t\left(X_{test}\right)$;

14:  Find non-dominated solutions $initPop$ in $X_{test}$;

15:  while $\left|initPop\right|>N$ do

16:       Delete individual in $initPop$;

17:  end while

18:  while $\left|initPop\right|<N$ do

19:       Add Gaussian noises to $initPop$;

20:  end while

21:  return $initPop$;

3.4. Computational Complexity

In CRTL-DMOEA, the computational costs are mainly spent on the process of clustering, non-dominated sorting, and regression transfer method. Clustering $PO{S}_{t-1}$ by localPCA consumes $O\left(d^2\right)$, where d is the dimension of decision variables. The complexity of the non-dominated sorting is $O(N^2\times M)$, where M is the number of objectives. In the regression transfer method, the computational complexity of obtaining the strong classifier is $O(N^2\times d)$.

4. Experimental Studies

4.1. Test Problems

In the experiment, we use the widely used FDA [45], dMOP [3], and F test suite [34] to evaluate all compared algorithms. The FDA test suit consists of five DMOPs, i.e., FDA1-FDA5, which are linearly related between decision variables. The dMOP test suite is proposed by extending the FDA test suite. Moreover, F5-F10 problems in F test suit have more complex dynamic geometries than others over time.

According to the different dynamical changes of DPOS and DPOF, DMOPs can be classified into four types:

• TYPE I: The POS changes over time, but the POF is fixed.
• TYPE II: Both the POS and POF change over time.
• TYPE III: The POS is fixed while the POF changes.
• TYPE IV: Both the POS and POF are fixed, but the problem changes.

Based on the classification mentioned above, FDA1, FDA4, and dMOP3 belong to Type I problem. FDA3, FDA5, dMOP2 and F5-F10 belong to Type II problem. FDA2 and dMOP1 belong to Type III problem. In these DMOPs, the time variable t is defined as:

$$t=(1/n_t)\lfloor \tau /{\tau }_t\rfloor$$

(11)

where $n_t$ and ${\tau }_t$ refer to the severity and frequency of changes. $\tau$ is the generation counter.

4.2. Compared Algorithms and Parameter Settings

The proposed CRTL-DMOEA is compared with four other popular algorithms, including MOEA/D-KF [46], PPS [34] Tr-MOEA/D [28] and KT-MOEA/D [30]. MOEA/D-KF and PPS are prediction-based DMOEAs, while Tr-MOEA/D and KT-MOEA/D are based on transfer learning. The algorithms are implemented in MATLAB R2020a on an Intel Core i7 with 2.70 GHz CPU on Windows 10. For a fair comparison, most parameters follow the original references. Other common parameters are summarized below:

• In the experiments, the population size N is set to 100 for biobjective optimization problems and 150 for triobjective problems. The number of decision variables n is set as 10.
• There are three pairs of dynamic configurations, which are ($n_t=5,{\tau }_t=10$), ($n_t=10,{\tau }_t=10$), and ($n_t=20,{\tau }_t=10$). The total number of generations is set to $20\times {\tau }_t$.
• In CRTL-DMOEA, the cluster number $N_c$ is 10, and MOEA/D [47] is used as the static optimizer.
• Each algorithm is run 20 times independently on each test problem.

4.3. Performance Metrics

4.3.1. Modified Inverted Generational Distance (MIGD)

The inverted generational distance (IGD) [33] is a commonly used metric to assess the performance of MOEAs in terms of convergence and diversity of the obtained solutions. A smaller IGD value indicates better convergence and higher diversity. Mathematically, the IGD is computed as:

$$\mathrm{IGD}(\mathrm{POF},{\mathrm{POF}}^{*})=\frac{1}{|{\mathrm{POF}}^{*}|}\sum _{p^{*}\in {\mathrm{POF}}^{*}}\underset{p\in \mathrm{POF}}{min}{\parallel p^{*}-p\parallel }^2$$

(12)

where ${\mathrm{POF}}^{*}$ is uniformly distributed points along the true POF. POF denotes the approximated POF obtained by a MOEA. Additionally, $\parallel p^{*}{-p\parallel }^2$ is the Euclidean distance between the point $p^{*}$ and p.

The MIGD metric is defined as the average of IGD values over all time steps over a run, i.e.,

$$\mathrm{MIGD}=\frac{1}{\left|T\right|}\sum _{t\in T}\mathrm{IGD}({\mathrm{POF}}_t,{\mathrm{POF}}_t^{*})$$

(13)

where T is a set of discrete time steps in a run.

4.3.2. Modified Hypervolume (MHV)

The hypervolume (HV) [25] is a metric that takes into account convergence and distribution of solutions simultaneously in order to evaluate the comprehensive quality of the obtained POF. A larger HV value indicates the better convergence and distribution. Like MIGD modified from IGD, MHV is defined as the average of the HV values in all time steps over a run.

4.4. Comparison with Other DMOEAs

In this section, we conduct the performance comparisons of all algorithms in solving different types of DMOPs, including FDA, dMOP and F problems described in Section 4.1. The statistical results on MIGD values obtained by compared algorithms are presented in

Table 1. MIGD values obtained by five algorithms on different test problems.

Problems	${\mathit{n}}_{\mathit{t}},{\mathit{\tau }}_{\mathit{t}}$	MOEA/D-KF	PPS	Tr-MOEA/D	KT-MOEA/D	CRTL-MOEA/D
FDA1	5,10	0.1482 ± 0.007(+)	0.3138 ± 0.031(+)	0.0215 ± 0.001(+)	0.0064 ± 0.013(=)	0.0053 ± 0.003
	10,10	0.1056 ± 0.001(+)	0.3584 ± 0.077(+)	0.0221 ± 0.004(+)	0.0055 ± 0.011(=)	0.0047 ± 0.002
	20,10	0.1049 ± 0.002(+)	0.3291 ± 0.016(+)	0.0197 ± 0.004(+)	0.0061 ± 0.016(=)	0.0047 ± 0.001
FDA2	5,10	0.1702 ± 0.011(+)	0.1442 ± 0.021(+)	0.0251 ± 0.002(+)	0.0599 ± 0.026(+)	0.0201 ± 0.002
	10,10	0.1448 ± 0.041(+)	0.1763 ± 0.039(+)	0.0347 ± 0.002(+)	0.0456 ± 0.023(+)	0.0302 ± 0.003
	20,10	0.1075 ± 0.018(+)	0.1557 ± 0.025(+)	0.0307 ± 0.004(+)	0.0319 ± 0.016(+)	0.0267 ± 0.003
FDA3	5,10	0.1804 ± 0.017(+)	0.2309 ± 0.007(+)	0.0254 ± 0.003(+)	0.0135 ± 0.004(=)	0.0108 ± 0.006
	10,10	0.1359 ± 0.009(+)	0.2229 ± 0.007(+)	0.0173 ± 0.001(+)	0.0088 ± 0.009(+)	0.0067 ± 0.003
	20,10	0.1586 ± 0.021(+)	0.2066 ± 0.002(+)	0.0207 ± 0.002(+)	0.0091 ± 0.011(=)	0.0071 ± 0.002
FDA4	5,10	0.1026 ± 0.003(+)	0.9421 ± 0.043(+)	0.1065 ± 0.003(+)	0.0812 ± 0.013(+)	0.0766 ± 0.003
	10,10	0.0986 ± 0.002(+)	0.9742 ± 0.135(+)	0.1113 ± 0.011(+)	0.0776 ± 0.016(+)	0.0734 ± 0.004
	20,10	0.1053 ± 0.009(+)	0.9436 ± 0.138(+)	0.1068 ± 0.001(+)	0.0782 ± 0.010(+)	0.0753 ± 0.007
FDA5	5,10	0.4484 ± 0.097(+)	0.4738 ± 0.078(+)	0.6072 ± 0.091(+)	0.3722 ± 0.118(-)	0.3830 ± 0.104
	10,10	0.4592 ± 0.084(+)	0.5680 ± 0.078(+)	0.6502 ± 0.092(+)	0.3695 ± 0.151(−)	0.3979 ± 0.101
	20,10	0.4321 ± 0.072(−)	0.6039 ± 0.067(+)	0.6425 ± 0.079(+)	0.5548 ± 0.179(+)	0.4958 ± 0.161
dMOP1	5,10	0.0238 ± 0.002(+)	0.4021 ± 0.007(+)	0.0136 ± 0.003(+)	0.0059 ± 0.007(+)	0.0055 ± 0.003
	10,10	0.0269 ± 0.001(+)	0.3934 ± 0.007(+)	0.0195 ± 0.004(+)	0.0060 ± 0.005(=)	0.0050 ± 0.002
	20,10	0.0310 ± 0.004(+)	0.4071 ± 0.010(+)	0.0187 ± 0.004(+)	0.0056 ± 0.002(+)	0.0051 ± 0.002
dMOP2	5,10	0.0843 ± 0.002(+)	0.3373 ± 0.017(+)	0.1122 ± 0.012(+)	0.0818 ± 0.011(+)	0.0366 ± 0.002
	10,10	0.0793 ± 0.001(+)	0.3236 ± 0.014(+)	0.1062 ± 0.017(+)	0.0751 ± 0.008(+)	0.0205 ± 0.003
	20,10	0.0714 ± 0.003(+)	0.3534 ± 0.026(+)	0.0981 ± 0.008(+)	0.0714 ± 0.012(+)	0.0084 ± 0.002
dMOP3	5,10	0.1281 ± 0.003(+)	0.3691 ± 0.011(+)	0.1261 ± 0.001(+)	0.1237 ± 0.014(+)	0.0049 ± 0.001
	10,10	0.1252 ± 0.002(+)	0.3526 ± 0.022(+)	0.1185 ± 0.001(+)	0.1236 ± 0.012(+)	0.0045 ± 0.002
	20,10	0.1225 ± 0.002(+)	0.3593 ± 0.007(+)	0.1288 ± 0.002(+)	0.1208 ± 0.012(+)	0.0052 ± 0.001
F5	5,10	1.4063 ± 0.193(+)	2.5053 ± 0.149(+)	0.5641 ± 0.076(+)	0.0787 ± 0.024(−)	0.1481 ± 0.012
	10,10	1.2651 ± 0.145(+)	2.2502 ± 0.076(+)	0.5767 ± 0.061(+)	0.0778 ± 0.018(=)	0.0757 ± 0.023
	20,10	1.4003 ± 0.021(+)	2.3815 ± 0.097(+)	0.5922 ± 0.082(+)	0.0799 ± 0.016(=)	0.0723 ± 0.021
F6	5,10	0.9273 ± 0.197(+)	3.2895 ± 0.267(+)	0.3089 ± 0.033(+)	0.2023 ± 0.019(+)	0.1884 ± 0.021
	10,10	0.9029 ± 0.162(+)	3.2479 ± 0.223(+)	0.3201 ± 0.053(+)	0.0962 ± 0.012(=)	0.0934 ± 0.015
	20,10	0.9313 ± 0.191(+)	3.3357 ± 0.161(+)	0.3305 ± 0.021(+)	0.0631 ± 0.014(−)	0.0760 ± 0.009
F7	5,10	0.9597 ± 0.053(+)	3.2057 ± 0.132(+)	0.3053 ± 0.021(+)	0.1779 ± 0.023(-)	0.1993 ± 0.014
	10,10	0.8627 ± 0.047(+)	3.2763 ± 0.081(+)	0.3068 ± 0.032(+)	0.0626 ± 0.017(+)	0.0547 ± 0.011
	20,10	0.8914 ± 0.159(+)	3.0467 ± 0.103(+)	0.2882 ± 0.011(+)	0.0668 ± 0.011(+)	0.0573 ± 0.021
F8	5,10	0.2634 ± 0.039(+)	1.1911 ± 0.146(+)	0.2739 ± 0.007(+)	0.3563 ± 0.223(+)	0.2469 ± 0.032
	10,10	0.2413 ± 0.002(=)	1.0986 ± 0.272(+)	0.2581 ± 0.006(+)	0.4248 ± 0.224(+)	0.2349 ± 0.011
	20,10	0.1951 ± 0.015(−)	1.1252 ± 0.335(+)	0.2823 ± 0.012(+)	0.3736 ± 0.184(+)	0.2373 ± 0.014
F9	5,10	0.5398 ± 0.022(+)	1.9148 ± 0.157(+)	0.3819 ± 0.022(+)	0.4421 ± 0.048(+)	0.3452 ± 0.032
	10,10	0.5455 ± 0.033(+)	1.6986 ± 0.161(+)	0.3520 ± 0.032(+)	0.1593 ± 0.056(=)	0.1549 ± 0.022
	20,10	0.5436 ± 0.064(+)	1.7181 ± 0.213(+)	0.3711 ± 0.036(+)	0.0745 ± 0.008(+)	0.0741 ± 0.008
F10	5,10	0.3008 ± 0.022(-)	1.5849 ± 0.134(+)	0.3615 ± 0.161(+)	0.3632 ± 0.097(+)	0.3397 ± 0.099
	10,10	3.0043 ± 0.055(−)	3.5599 ± 0.163(+)	3.4388 ± 0.419(+)	3.4258 ± 0.007(+)	3.2806 ± 0.009
	20,10	2.9791 ± 0.089(−)	3.6534 ± 0.411(+)	3.8577 ± 0.325(+)	3.2290 ± 0.107(−)	3.5242 ± 0.056
$+/=/-$		36/1/5	42/0/0	42/0/0	26/10/6

. In this table, the symbols $(+)$ and $(-)$ denote that the proposed CRTL-MOEA/D performs significantly better and worse than compared algorithm, respectively, while the symbol $(=)$ means there is no significant difference by utilizing the Wilcoxon rank sum test [48] at a significance level of 0.05.

As can be seen from Table 1, CRTL-MOEA/D achieves 32 out of 42 best results on FDA, dMOP and F test problems in terms of MIGD metric. CRTL-MOEA/D shows slightly worse than KT-MOEA/D on F5, F7 and F8. However, CRTL-MOEA/D performs significantly worse on FDA5 and F10 problems, which is possibly due to the complex characteristics of these problems. In FDA5, both the geometric shapes of POF and POS are not fixed in dynamic environment. In F10, the POS occasionally jumps from one area to another one and two adjacent POFs are different. The complications make it difficult to acquire valid historical knowledge for building a regression transfer model to generate a high-quality initial population when the change occurs.

To visually show the comprehensive performance of all algorithms at different environments, we plot the average logarithmic IGD in the first 20 changes with $n_t=10,{\tau }_t=10$ in Figure 3. It is clear to see that, compared with other algorithms, CRTL-MOEA/D achieves better IGD results and stability with time.

Besides, to qualify the significant differences, Figure 4 gives the average performance rankings with Friedman test [49] with regard to MIGD. A lower average performance score indicates a better overall performance. As observed from Figure 4, CRTL-MOEA/D has the best score 1.40, indicating the better performance than MOEA/D-KF (3.35), PPS (4.71), Tr-MOEA/D (3.29), and KT-MOEA/D (2.25). Moreover, we take another post-hoc Nemenyi test [49] to plot the critical difference (CD) in Figure 5, which shows the significance of paired differences among all algorithms. It shows that CRTL-MOEA/D is only comparable to KT-MOEA/D while significantly different from others. Besides, MHV results are provided in

Table 2. MHV values obtained by five algorithms on different test problems.

Problems	${\mathit{n}}_{\mathit{t}},{\mathit{\tau }}_{\mathit{t}}$	MOEA/D-KF	PPS	Tr-MOEA/D	KT-MOEA/D	CRTL-MOEA/D
FDA1	(5,10)	0.6111 ± 0.045(+)	0.6103 ± 0.037(+)	0.6634 ± 0.036(+)	0.7475 ± 0.011(+)	0.7702 ± 0.012
	(10,10)	0.6571 ± 0.115(+)	0.5912 ± 0.045(+)	0.7029 ± 0.061(+)	0.6771 ± 0.014(+)	0.8271 ± 0.009
	(20,10)	0.6515 ± 0.055(+)	0.5858 ± 0.026(+)	0.6678 ± 0.056(+)	0.7345 ± 0.009(+)	0.7994 ± 0.011
FDA2	(5,10)	0.5600 ± 0.114(+)	0.6349 ± 0.022(+)	0.6099 ± 0.112(+)	0.6587 ± 0.012(+)	0.6726 ± 0.015
	(10,10)	0.5624 ± 0.073(+)	0.5963 ± 0.060(+)	0.6147 ± 0.126(+)	0.6123 ± 0.015(+)	0.6647 ± 0.009
	(20,10)	0.5817 ± 0.037(+)	0.6040 ± 0.050(+)	0.6306 ± 0.123(+)	0.6554 ± 0.009(=)	0.6526 ± 0.031
FDA3	(5,10)	0.5014 ± 0.121(+)	0.7764 ± 0.042(+)	0.5258 ± 0.129(+)	0.5403 ± 0.041(+)	0.5762 ± 0.039
	(10,10)	0.5197 ± 0.129(+)	0.7955 ± 0.095(+)	0.4907 ± 0.112(+)	0.5961 ± 0.021(=)	0.5947 ± 0.066
	(20,10)	0.4924 ± 0.053(+)	0.8093 ± 0.116(+)	0.5059 ± 0.025(+)	0.5844 ± 0.012(+)	0.6066 ± 0.011
FDA4	(5,10)	0.6591 ± 0.022(+)	0.0723 ± 0.004(+)	0.6134 ± 0.029(+)	0.6255 ± 0.164(+)	0.6948 ± 0.098
	(10,10)	0.6499 ± 0.118(+)	0.0404 ± 0.002(+)	0.6405 ± 0.115(+)	0.6177 ± 0.152(+)	0.6614 ± 0.121
	(20,10)	0.6787 ± 0.065(+)	0.0398 ± 0.002(+)	0.5698 ± 0.113(+)	0.6215 ± 0.177(+)	0.7198 ± 0.053
FDA5	(5,10)	0.3677 ± 0.016(=)	0.1509 ± 0.007(+)	0.2983 ± 0.088(+)	0.3206 ± 0.113(+)	0.3629 ± 0.111
	(10,10)	0.3559 ± 0.027(+)	0.1479 ± 0.029(+)	0.3149 ± 0.023(+)	0.3734 ± 0.028(+)	0.3811 ± 0.019
	(20,10)	0.2952 ± 0.012(=)	0.1280 ± 0.011(+)	0.2539 ± 0.021(+)	0.2872 ± 0.119(=)	0.2822 ± 0.089
dMOP1	(5,10)	0.6337 ± 0.098(+)	0.3641 ± 0.007(+)	0.5820 ± 0.119(+)	0.6695 ± 0.022(+)	0.6977 ± 0.029
	(10,10)	0.6317 ± 0.055(+)	0.3690 ± 0.028(+)	0.5597 ± 0.168(+)	0.6368 ± 0.019(+)	0.7223 ± 0.031
	(20,10)	0.6247 ± 0.218(+)	0.3551 ± 0.021(+)	0.5793 ± 0.032(+)	0.6317 ± 0.014(+)	0.6907 ± 0.018
dMOP2	(5,10)	0.6836 ± 0.039(+)	0.3572 ± 0.024(+)	0.6373 ± 0.069(+)	0.6476 ± 0.021(+)	0.7002 ± 0.015
	(10,10)	0.6687 ± 0.088(+)	0.3527 ± 0.066(+)	0.6014 ± 0.099(+)	0.6185 ± 0.029(+)	0.6995 ± 0.029
	(20,10)	0.6839 ± 0.049(+)	0.3811 ± 0.041(+)	0.6105 ± 0.103(+)	0.6413 ± 0.019(+)	0.7329 ± 0.017
dMOP3	(5,10)	0.6957 ± 0.091(+)	0.6767 ± 0.034(+)	0.6700 ± 0.032(+)	0.7124 ± 0.029(+)	0.7911 ± 0.032
	(10,10)	0.7655 ± 0.104(-)	0.6740 ± 0.018(+)	0.6477 ± 0.041(+)	0.7464 ± 0.049(=)	0.7469 ± 0.051
	(20,10)	0.6954 ± 0.108(+)	0.6885 ± 0.017(+)	0.6499 ± 0.129(+)	0.7447 ± 0.025(+)	0.8166 ± 0.029
F5	(5,10)	0.1843 ± 0.022(+)	0.0187 ± 0.003(+)	0.4422 ± 0.066(+)	0.6800 ± 0.028(−)	0.6684 ± 0.037
	(10,10)	0.1916 ± 0.038(+)	0.0154 ± 0.001(+)	0.4229 ± 0.055(+)	0.6962 ± 0.018(+)	0.7130 ± 0.022
	(20,10)	0.1843 ± 0.038(+)	0.0192 ± 0.003(+)	0.4361 ± 0.049(+)	0.6467 ± 0.031(+)	0.7024 ± 0.051
F6	(5,10)	0.1829 ± 0.027(+)	0.0094 ± 0.001(+)	0.5030 ± 0.017(+)	0.5400 ± 0.061(+)	0.6002 ± 0.041
	(10,10)	0.1871 ± 0.050(+)	0.0094 ± 0.003(+)	0.5112 ± 0.108(+)	0.6550 ± 0.011(+)	0.6704 ± 0.020
	(20,10)	0.1753 ± 0.041(+)	0.0106 ± 0.002(+)	0.5122 ± 0.130(+)	0.6811 ± 0.049(=)	0.6925 ± 0.032
F7	(5,10)	0.1985 ± 0.027(+)	0.0023 ± 0.003(+)	0.4851 ± 0.056(+)	0.5833 ± 0.008(+)	0.6185 ± 0.011
	(10,10)	0.1902 ± 0.041(+)	0.0029 ± 0.006(+)	0.5012 ± 0.055(+)	0.6285 ± 0.018(+)	0.6532 ± 0.009
	(20,10)	0.2246 ± 0.038(+)	0.0029 ± 0.001(+)	0.5349 ± 0.073(+)	0.6330 ± 0.025(+)	0.7603 ± 0.025
F8	(5,10)	0.6722 ± 0.050(=)	0.0185 ± 0.003(+)	0.6103 ± 0.036(+)	0.6141 ± 0.032(+)	0.6649 ± 0.027
	(10,10)	0.6490 ± 0.091(=)	0.0148 ± 0.003(+)	0.5808 ± 0.171(+)	0.6036 ± 0.071(+)	0.6588 ± 0.066
	(20,10)	0.6700 ± 0.135(=)	0.0125 ± 0.002(+)	0.5926 ± 0.072(+)	0.6190 ± 0.095(=)	0.6337 ± 0.119
F9	(5,10)	0.3659 ± 0.057(+)	0.0363 ± 0.005(+)	0.4447 ± 0.079(+)	0.4698 ± 0.025(+)	0.5003 ± 0.021
	(10,10)	0.3159 ± 0.021(+)	0.0481 ± 0.002(+)	0.4378 ± 0.043(+)	0.5590 ± 0.015(+)	0.6077 ± 0.029
	(20,10)	0.3691 ± 0.015(+)	0.0441 ± 0.005(+)	0.4688 ± 0.057(+)	0.6494 ± 0.031(+)	0.6885 ± 0.036
F10	(5,10)	0.3371 ± 0.033(+)	0.0328 ± 0.004(+)	0.4537 ± 0.034(+)	0.4520 ± 0.036(+)	0.4892 ± 0.042
	(10,10)	0.5449 ± 0.061(+)	0.4505 ± 0.051(+)	0.6040 ± 0.023(+)	0.6710 ± 0.031(+)	0.6838 ± 0.032
	(20,10)	0.5848 ± 0.063(+)	0.4597 ± 0.012(+)	0.6191 ± 0.023(+)	0.7414 ± 0.029(−)	0.7169 ± 0.027
$+/=/-$		36/5/1	42/0/0	42/0/0	34/6/2

. From all the experimental results, we can conclude that CRTL-DMOEA is more effective than other algorithms to track the time-varying POS in terms of convergence and diversity on most test cases.

4.5. Ablation Study

The proposed CRTL-MOEA/D has two key components: cluster-based selection and cluster-based regression transfer. To validate the effectiveness of these two mechanisms, an ablation experiment is carried out by comparing CRTL-MOEA/D with two variants (CRTL-MOEA/D${}_C$ and CRTL-MOEA/D${}_T$) on different problems with $n_t=10$ and ${\tau }_t=10$. CRTL-MOEA/D${}_C$ utilizes the estimated clustering centroids and random solutions to form the initial population without regression transfer learning. In CRTL-MOEA/D${}_T$, the centriod selection is removed. The previous POSs and their objective values are treated as the source domain for regression transfer. The random solutions generated in the new environment and objective values are used as the target domain.

The statistical results on MIGD valus are provided in

Table 3. MIGD values obtained by CRTL-MOEA/D${}_C$, CRTL-MOEA/D${}_T$ and CRTL-MOEA/D.

Problems	CRTL-MOEA/D${}_{\mathit{C}}$	CRTL-MOEA/D${}_{\mathit{T}}$	CRTL-MOEA/D
FDA1	0.0051 ± 0.003(=)	0.0055 ± 0.002(=)	0.0047 ± 0.002
FDA2	0.0401 ± 0.003(+)	0.0413 ± 0.009(+)	0.0302 ± 0.003
FDA3	0.0096 ± 0.004(+)	0.0081 ± 0.002(=)	0.0067 ± 0.003
FDA4	0.0812 ± 0.003(+)	0.0712 ± 0.005(−)	0.0734 ± 0.004
FDA5	0.3711 ± 0.135(−)	0.4587 ± 0.081(+)	0.3979 ± 0.101
dMOP1	0.0065 ± 0.002(=)	0.0074 ± 0.003(+)	0.0050 ± 0.002
dMOP2	0.0240 ± 0.001(+)	0.0226 ± 0.009(+)	0.0205 ± 0.003
dMOP3	0.0071 ± 0.001(+)	0.0058 ± 0.001(=)	0.0045 ± 0.002
+/=/−	5/2/1	4/3/1

. From Table 3, it is clear to see that CRTL-MOEA/D shows superior performance over CRTL-MOEA/D${}_C$ on most cases, indicating that the cluster-based regression transfer contributes to exploit informative historical knowledge to generate a high-quality initial population for the new environment. In addition, CRTL-MOEA/D surpasses CRTL-MOEA/D${}_T$ on most problems. Thus, we can conclude that it is more effective to transfer clustering centroids than all the non-dominated solutions. In addition, CRTL-MOEA/D obtains the best results in 6 out of 8 cases, which also shows the effectiveness of combining these two components.

4.6. Running Time

In this section, we compare the running time of different algorithms and provide the results in

Table 4. Running time obtained by five algorithms on F problems.

Problems	MOEA/D-KF	PPS	Tr-MOEA/D	KT-MOEA/D	CRTL-MOEA/D
F5	6.3234	5.2454	65.0048	1.3050	1.0953
F6	5.2214	4.9055	68.1323	1.2645	0.9222
F7	7.0051	6.1212	69.0055	1.3339	1.1041
F8	11.0158	9.2353	111.0768	4.2986	4.0387
F9	8.0520	7.1115	88.0027	1.2630	0.9930
F10	6.6856	5.6315	62.5832	1.1533	1.2052

. As observed in the table, CRTL-DMOEA has smaller running time than other algorithms on most of test instances. This shows that the proposed cluster-based selection and regression transfer method are very efficient. Tr-MOEA/D needs more running time than CRTL-DMOEA. The main reason behind this is that CRTL-DMOEA selects the representative individuals to transfer, rather than all optimal solutions used in Tr-MOEA/D, which can save a lot of running time. Besides, Tr-MOEA/D needs more parameters to build the latent space, which takes $O(N^3\times L)$, where L is the total number of bits of the input. However, CRTL-DMOEA constructs an essentially sample-based regression model, which can avoid the need for more parameter settings and improve the computational complexity. To summarize, CRTL-DMOEA seems competitive with others on most test instances in terms of computational efficiency.

5. Conclusions

Transfer learning-based DMOEAs have been shown to be effective for solving DMOPs, but most of them suffer from some issues: transferring a large number of common solutions consumes too much in terms of resources and probably causes negative transfer; knowledge transfer through a latent space requires more parameters and takes an excessive amount of time.

To overcome the challenges, a cluster-based regression transfer learning method-based DMOEA, called CRTL-DMOEA, has been proposed in this paper. In CRTL-DMOEA, the cluster-based selection mechanism was first applied to find the centroids of approximate POSs, which were then estimated with an AR prediction model. Subsequently, a cluster-based regression transfer was introduced to build an regression transfer model based on TrAdaboost.R2, by exploiting the knowledge from the clustering centroids. Then, the regression transfer model was used to generate the high-quality initial population for a new environment. By comparing with four other popular DMOEAs and two variants of CRTL-DMOEA, CRTL-DMOEA has demonstrated to be able to effectively track the changing POS/POF over time.

In future, we are interested in utilizing different transfer methods to efficiently solve DMOPs. Furthermore, we will try to apply the proposed method to solve some real-life DMOPs.