*4.3. Inter-Population Reproduction*

The major function of inter-population reproduction is knowledge transfer between different subpopulations, which may help to accelerate the search process and find global solutions [51]. Therefore, when, what, and how to transfer are the key issues in MTEC. An excellent MTEC algorithm should be able to deal with the three problems properly [102].

#### 4.3.1. When to Transfer

As depicted in Figure 6, inter-population reproduction can happen at any stage of the optimization process in a multi-task scenario. Generally, the offspring are generated via genetic transfer (crossover and mutation) across tasks for each generation in [18].

In fact, knowledge transfer across tasks can also occur with a fixed generation interval along the evolution search. The interval of inter-population reproduction was set to 10 generations in EMT (evolutionary multitasking) [21], and the generation interval was fixed at 20 generations in SGDE [102]. Experimental results based on the island model revealed that better results are observed from small transfer intervals than from large transfer intervals [103].

Due to the essential differences among the landscapes of the optimization tasks, Wen and Ting [104] suggested stopping the information transfer when the parting way is detected. In MT-CPSO, if a particle within a particular population did not improve its personal best position over prescribed consecutive generations, knowledge acquired from the other task was transferred across to assist the search in more promising regions [53]. Obviously, the greater the value of the prescribed iterations is, the smaller the probability of inter-population reproduction is. Similarly, in SOMAMIF, the current optimal fitness of each population was firstly judged, and the knowledge transfer demand across tasks was triggered when the evolution process of a task stagnated for successive generations [97].

#### 4.3.2. What to Transfer

In MFEA and its variants, each solution in every task will be selected as a transferred solution based on the same probability. The light-weight knowledge transfer strategy was proposed by Zheng et al. [105]. To be more specific, the best solutions found so far on transfer other tasks to the given task and randomly replace some individuals during the optimization process.

However, some transferred solutions, even the best solutions found so far, do not help to optimize the other tasks, thereby leading to the low efficiency of achieving the positive transfer. In evolutionary multi-task via explicit autoencoding, transferred solutions are selected from the nondominated solutions in each task [21], while the performance of this method may primarily rely on the high degree of underlying intertask similarities [41]. Recently, Lin et al. [19] proposed a new strategy for selecting valuable solutions for positive transfer. In the proposed approach, a transferred solution achieves positive transfer if it is nondominated in its target task. Then, in the original search space of this positive-transfer

solution, its several closest (based on the Euclidean distance) solutions will turn into the transferred solutions, since these solutions are more likely to achieve positive transfer.

In the existing DE-based on MTEC, the knowledge is transferred only by randomly selecting the solutions from different tasks to generate offspring without regarding the search property of DE. In fact, the successful difference vectors from the past generations can not only retain the important landscape information of the optimization problem, but also preserve the population diversity during the evolutionary process. Motivated by this consideration, Cai et al. [87] proposed a difference vector sharing mechanism for DEbased MTEC, aiming at capturing, sharing, and utilizing the knowledge of the promising difference vectors found in the evolutionary process.

More recently, Lin et al. [106] have utilized incremental Naive Bayes classifiers to select valuable solutions to be transferred during multi-task search, thus leading to the promising convergence of tasks. Furthermore, under the existing mapping strategies, tasks may be trapped in local Pareto Fronts with the guide of knowledge transfer. Thus, with the aim of improving overall convergence behavior, a randomized mapping among tasks is added that enhances the exploration capacity of transferred solutions.

Zhou et al. [107] investigated what information, except to the selective individuals, should be transferred in an MFEA framework. In particular, the difference between the individual solution and the estimated optimal solution, called the individual gradient (IG), was introduced as the additional knowledge to be transferred. The proposed approach was applied to mobile agen<sup>t</sup> path planning (MAPP) [107] and the autonomous underwater vehicles (AUV) 3D path planning problem [108].

Based on a novel idea of multiproblem surrogates (MPS), an adaptive knowledge reuse framework was proposed for surrogate-assisted multi-objective optimization of computationally expensive problems [109]. The MPS provides the capability of acquiring and spontaneously transferring learned models gained from distinct but possibly related problem-solving experiences. The proposed framework consists of four primary steps: initialization, aggregation, multi-problem surrogate, and evolutionary optimization. The authors further present one possible instantiation, which utilizes a Tchebycheff aggregation approach, Gaussian process surrogate models with linear meta-regression, and an expected improvement measure to quantify the merit of evaluating a new point.

#### 4.3.3. How to Knowledge Transfer Implicitly

As the most natural way, knowledge transfer across tasks is realized implicitly when two individuals possessing different skill factors are selected for generating the offspring via crossover. The implicit MTEC usually employs a single population with unified solution representation to solve multiple optimization tasks.

Compared with single-population SBX crossover, two parents come from two different subpopulations ( *Pk* and *Pr*). Take MFEA as an example, knowledge transfer is done by inter-population SBX crossover as below [18]:

$$\mathbf{x}\_{i\*}^{k}\text{ or }\mathbf{x}\_{i\*}^{r} = \begin{cases} 0.5 \Big( (1+\gamma)\mathbf{x}\_{i}^{k} + (1-\gamma)\mathbf{x}\_{j}^{r} \Big), \quad rand \le 0.5\\ 0.5 \Big( (1+\gamma)\mathbf{x}\_{j}^{r} + (1-\gamma)\mathbf{x}\_{i}^{k} \Big), \quad rand > 0.5 \end{cases} \tag{4}$$

For MT-CPSO (multitasking coevolutionary particle swarm optimization), the interpopulation reproduction is provided as follows [88,92,93]:

$$\mathbf{x}\_{i\*}^{k} = 0.5 \left( (1 + rand)\mathbf{x}\_{i}^{k} + (1 - rand)\mathbf{x}\_{\mathcal{S}^{b}}^{r} \right) \tag{5}$$

where *xk i* and *xk i*∗ are the position of the *i*-th particle and its corresponding updated particle in subpopulation *Pk*, respectively, *xrgb* is the current global best position in subpopulation *Pr*, and *rand* is a random number between 0 and 1.

To explore the generality of MFEA with different search mechanisms, Feng et al. [85] investigated two MTEC approaches by using PSO and DE as the search engine, respectively. While the other genetic operators are kept the same as the original MFEA, the velocity is updated for MFPSO (multifactorial particle swarm optimization) using the following equation [85]:

$$
\boldsymbol{\sigma}\_{i\*}^{k} = \boldsymbol{\omega} \cdot \boldsymbol{\sigma}\_{i}^{k} + \boldsymbol{c}\_{1} \cdot \boldsymbol{rand} \cdot (\mathbf{x}\_{lb}^{k} - \mathbf{x}\_{i}^{k}) + \boldsymbol{c}\_{2} \cdot \boldsymbol{rand} \cdot (\mathbf{x}\_{\mathcal{g}b}^{k} - \mathbf{x}\_{i}^{k}) + \boldsymbol{c}\_{3} \cdot \boldsymbol{rand} \cdot (\mathbf{x}\_{\mathcal{g}b}^{r} - \mathbf{x}\_{i}^{k}).\tag{6}
$$

For MFDE (multifactorial differential evolution), the mutation operator with genetic materials transfer is defined as following [85]:

$$\mathbf{x}\_{i\*}^{k} = \mathbf{x}\_{r1}^{k} + F\_i(\mathbf{x}\_{r2}^{r} - \mathbf{x}\_{r3}^{r}). \tag{7}$$

For AMFPSO (adaptive multifactorial particle swarm optimization), the velocity is updated using the following equation [94]:

$$\boldsymbol{\sigma}\_{i\*}^{k} = \boldsymbol{\omega} \cdot \boldsymbol{\sigma}\_{i}^{k} + \boldsymbol{c}\_{1} \cdot \text{rand} \cdot (\mathbf{x}\_{lb}^{k} - \mathbf{x}\_{i}^{k}) + \boldsymbol{c}\_{2} \cdot \text{rand} \cdot (\mathbf{x}\_{gb}^{k} - \mathbf{x}\_{i}^{k}) + \boldsymbol{c}\_{3} \cdot \text{rand} \cdot (\mathbf{x}\_{r1}^{l} - \mathbf{x}\_{r2}^{l}) \tag{8}$$

where *vki* and *vki*∗ are the velocity of the *i*-th particle and its corresponding updated particle in subpopulation *Pk*, respectively, *xki* and *xklb* are the position of the *i*-th particle and its best found-so-far particle in subpopulation *Pk*, respectively, *xkgb* is the current global best position in subpopulation *Pk*, *r*1 and *r*2 are random and mutually exclusive integers, *c*1, *c*2, *c*3, and *ω* are four parameters to adapt to problems, and *rand* is a random number within 0 and 1.

Recently, Song et al. [90] proposed a multitasking multi-swarm optimization (MTMSO) algorithm, in which knowledge transfer across tasks was realized via arithmetic crossover on the personal best *xbestki*of each particle among different tasks for every generation.

$$\mathbf{x} \mathbf{b} \mathbf{s} \mathbf{t}\_{i\*}^{k} = (1 - rand) \cdot \mathbf{x} \mathbf{b} \mathbf{s} \mathbf{t}\_{i}^{k} + rand \cdot \mathbf{x} \mathbf{b} \mathbf{s} \mathbf{t}\_{j}^{r} \tag{9}$$

For MPEF-SHADE (multi-population evolution framework—success-history based adaptive DE), the mutation operator with genetic materials transfer is defined as following [82,83]:

$$\mathbf{x}\_{i\*}^{k} = \mathbf{x}\_{i}^{k} + F\_{i} \left(\mathbf{x}\_{\mathcal{S}^{b}}^{r} - \mathbf{x}\_{i}^{k}\right) + F\_{i} (\mathbf{x}\_{r1}^{r} - \mathbf{x}\_{r2}^{r}) \tag{10}$$

where *xki* and *xki*∗ are the *i*-th individual and the corresponding updated individual in subpopulation *Pk*, respectively, *xrgb* is the current best individual in subpopulation *Pr*, *Fi* is the scaling factor, and *r*1 and *r*2 are random and mutually exclusive integers.

The transfer spark was proposed to exchange information between different tasks in MTO-FWA [96]. The core idea is to bind a firework and its generated explosion sparks and guiding sparks into a task module to solve a specific problem. Based on this, assume the *i*th firework for the optimization task *k* is denoted as *FWki* and the transfer spark generated by *FWki* under the guiding of *TVkj i* is represented as *TSkj i* . Therefore, *TVkj i* and *TSkj i* can be obtained by Equations (11) and (12), respectively

$$TV\_i^{kj} = \frac{2}{\sigma M\_k + \sigma M\_j} (\sum\_{i=1}^{\sigma M\_j} \mathbf{x}\_i^j - \sum\_{i=1}^{\sigma M\_k} \mathbf{x}\_i^k) \frac{r^{-\alpha}}{\sum\_{r=1}^{N\_k} r^{-\alpha}} \tag{11}$$

$$\mathbf{TS}\_{i}^{kj} = \mathbf{FW}\_{i}^{k} + \mathbf{TV}\_{i}^{kj} \tag{12}$$

where *Mk* and *Mj* denote the total number of the individuals that the skill factor is *k* and *j*, respectively.

In order to enhance knowledge transfer among different tasks, Yin et al. [110] integrated a new cross-task knowledge transfer as following, which used a search direction from another task

$$\mathbf{x}\_{i\*}^{k} = \mathbf{x}\_{\text{ellitr}}^{k} + (\mathbf{x}\_{i}^{r} - \mathbf{x}\_{\text{ellitc}}^{r}) \tag{13}$$

where *xkelite* and *xrelite* are the elite individuals of task *k* and *r*, respectively. The elite individual of the task is used to speed up the population convergence and the difference vector from another task can enhance the search diversity.

In EMT-RE framework for large-scale optimization, the knowledge transfer across tasks was conducted implicitly through the chromosomal crossover with two solutions possessing different skill factors [111]. If the current task is exactly the original task, the mutant chromosome *v<sup>p</sup> i*is simply generated from intermediate vector *ui* by:

$$
\boldsymbol{\sigma}\_{i}^{p} = \boldsymbol{\sigma}\_{r1}^{p} + F\_{i} \boldsymbol{u}\_{i} \tag{14}
$$

where *v<sup>p</sup> r*1 is a randomly chosen individual from the current task, and *Fi* is the differential weight for controlling the amplitude of difference. If not, *ui* will be mapped into the embedded space of the current task by the pseudo inverse of random embedding matrix *<sup>p</sup>inv*(*Ap*):

$$\boldsymbol{\sigma}\_{i}^{p} = \boldsymbol{\sigma}\_{r1}^{p} + F\_{i}(\boldsymbol{pinv}(\boldsymbol{A}\_{p})\boldsymbol{u}\_{i}) \tag{15}$$

where *pinv*(*A*) is approximated by *ATA* −1 *AT*.

Under the existing mapping strategies, tasks may be trapped in local Pareto Fronts with the guide of the knowledge transfer. Thus, with the aim of improving overall convergence behavior, a randomized mapping among tasks was added as follows, that enhances the exploration capacity of transferred solutions [106].

$$\mathbf{x}' = \begin{cases} \frac{(\mathcal{U}\_k - L\_k)(\mathbf{x} - L\_i)}{\mathcal{U}\_l - L\_i} + L\_k, & r > p\\ \frac{(\mathcal{U}\_k - L\_k)(\mathbf{x} - L\_i)}{\mathcal{U}\_l - L\_i} + \lambda (\mathcal{U}\_k - L\_k), & \text{otherwise} \end{cases} \tag{16}$$

where *λ* ∼ *<sup>U</sup>*[*<sup>a</sup>*, *b*], *r* ∼ *U*[0, 1], and *p* ∈ [0, 1], which controls the probability of exploring the search space.

#### 4.3.4. How to Knowledge Transfer Explicitly

In contrast to the existing implicit MTEC, the explicit MTEC algorithm employs an independent population for each optimization task and conducts knowledge transfer across tasks in an explicit manner. There are several advantages of explicit MTEC [112]. First, since each task has separate population for evolution, task-specific solution encoding schemes are employed for different tasks. Next, by only designing an explicit knowledge transfer operator, the explicit MTEC paradigm can be easily developed by employing different existing evolutionary solvers with various search capabilities for each optimization task. As different search mechanisms possess various search biases, the employment of problemspecific search operators in explicit MTEC could lead to a significantly improved algorithm performance. Further, rather than probabilistically selecting solutions for mating across tasks in the implicit MTEC, more flexible solution selection schemes, such as elite selection, can be performed before transfer in the explicit EMT for reducing negative knowledge transfer effects. However, compared with the accomplishments made in the implicit MTEC algorithms, only a few attempts have been conducted for developing the explicit MTEC approaches.

As a pioneering work, Bali et al. [113] put forward an MFEA variant with a linearized domain adaptation strategy, named LDA-MFEA, for transforming the search space of a simple task into its constitutive complex task which possesses a similar search space. The goal is to alleviate the negative transfer and to improve the quality of the generated offspring.

Feng et al. [21,114] developed an explicit MTEC algorithm to learn optimal linear mappings between different multiobjective tasks using a denoising autoencoder. In this method, different evolutionary mechanisms with different biases are cooperatively applied to solve various tasks simultaneously and the learned mappings serve as a bridge between tasks so that adaptive knowledge transfers can be conducted. By configuring the input and output layers to represent two task domains, the hidden representation provides a possibility for conducting knowledge transfer across task domains. In particular, let *P* and *Q* represent the set of solutions uniformly and independently sampled from the search space of two different tasks *T*1 and *T*2, respectively. Then the mapping *M* from *T*1 to *T*2 is given by

$$\mathcal{M} = \left(\boldsymbol{Q}\boldsymbol{P}^{\mathrm{T}}\right)\left(\boldsymbol{P}\boldsymbol{P}^{\mathrm{T}}\right)^{-1}.\tag{17}$$

Therefore, the optimized solutions found for different tasks along the evolutionary search can be explicitly transferred across tasks via a simple matrix multiplication operation with the learned *M*. The authors further improved the explicit knowledge transfer to address combinatorial optimization problems, such as VRPs [112]. In particular, they developed two mechanisms: the weighted *l*1-norm-regularized learning process for capturing the transfer mapping and the solution-based knowledge transfer process across VRPs.

Aiming to strengthen the knowledge transfer efficiency, a novel genetic transform strategy was proposed and applied in individual reproduction [22]. Given two tasks *T*1 and *T*2, two mapping vectors *M*12 (from *T*1 to *T*2) and *M*21 (from *T*2 to *T*1) are calculated as follows:

$$\mathcal{M}\_{21} = \left( \mathsf{mean}\_{T\_1} + \varepsilon \right) ./ \left( \mathsf{mean}\_{T\_2} + \varepsilon \right) \tag{18}$$

$$\mathcal{M}\_{12} = \left( \mathsf{mean}\_{T\_2} + \varepsilon \right) ./ \left( \mathsf{mean}\_{T\_1} + \varepsilon \right) \tag{19}$$

where *meanT*1 and *meanT*2 are mean vectors of some selected individuals specific to the two tasks, respectively, and *ε* represents a small positive number. The operator performs element-wise division of two vectors. Based on two vectors, the parent individuals can be mapped to the vicinity of the other solutions.

It was very recently determined that a novel search space mapping mechanism, namely, subspace alignment (SA) could enable efficient and high-quality knowledge transfer among different tasks [115]. In particular, the SA strategy establishes the connection between two tasks using two transforming matrices, which can reduce the probability of negative transfer. This involves assuming there are two subpopulations *P* and *Q***,** with each associated with a task. They denote the source data and target data, respectively. *WP* = 1 *nPT<sup>P</sup>* and *<sup>W</sup>Q* = 1 *nQ<sup>T</sup>Q* denote the covariance matrices of *P* and *Q*, respectively. Then *EP* and *EQ* consist of the set of all eigenvectors of *WP* and *WQ*, respectively, with one eigenvector per column. From *EP* and *EQ*, the eigenvectors corresponding to the largest *h* eigenvalues that can retain 95% of the information are selected to construct the subspaces of *P* and *Q*, that is, *SP* and *SQ*. Afterward, the transformation matrix *M*∗ of mapping *SP* and *SQ* is obtained according to Equation (20).

$$\mathbf{M}^\* = \mathbf{S}\_P \mathbf{}^T \mathbf{S}\_Q \tag{20}$$

The transferability between two distinct tasks is effectively enhanced with a proper domain adaptation technique. However, the improper pairwise learning fashion may incur a chaotic matching problem, which dramatically degrades the inter-task mapping [110]. Keeping this in mind, a novel rank loss function for acquiring a superior inter-task mapping between the source-target instances was formulated [116]. Then, an evolutionary-pathbased probabilistic representation model was proposed to represent the optimization instances. With the proposed representation model, the threat of chaotic matching between the source-target domains is effectively avoided. Finally, with a progressional Gaussian representation model, a closed-form solution of affine transformation for bridging the gap between the source-target instances was mathematically derived from the proposed rank loss function.

Recently, Chen et al. [117] proposed an evolutionary multi-task algorithm with learning task relationships (LTR) for the MOO problem. The decision space of each task is treated as a manifold, and all decision spaces of different tasks are jointly modeled as a joint manifold. The joint mapping matrix composed of multiple mapping functions is then constructed to map the decision spaces of different tasks to the latent space. Finally, the

relationships among distinct tasks can be jointly learned so as to promote the optimizing of all the tasks in a MOO problem.

Similarly, Tang et al. [42] also introduced an inter-task knowledge transfer strategy. Specifically, the low-dimension subspaces of task-specific decision spaces are first established via the principal component analysis (PCA) method. Then, the alignment matrix between two subspaces is learned and solved. After that, the corresponding solutions belonging to different tasks are projected into the subspaces. With this, two inter-task reproduction strategies are then designed in the aligned subspaces.

#### *4.4. Balance between Intra-Population Reproduction and Inter-Population Reproduction*

As illustrated in Figure 6, the offspring of individuals are generated in two ways: intrapopulation reproduction and inter-population reproduction. On one hand, the inductive biases transferred from another task are helpful to effectively accelerate convergence. On the other hand, excessive inter-population reproduction may lead to negative genetic transfer across tasks and bad algorithm performance [11,118]. Thus, a natural question in multi-task optimization community is finding a proper balance between intra-population reproduction and inter-population reproduction [51]. Up to now, the proposed approaches have been divided into three groups (fixed parameter, parameter adaptation, and resource reallocating) explained in the following subsections.

#### 4.4.1. Fixed Parameter Strategy

In the original MFEA, the extent of inter-task knowledge transfer is mandated by a scalar parameter defined as the random mating probability (*rmp*), which is set as a constant of 0.3 [18]. A larger value of *rmp* induces more exploration of the entire search space, thereby facilitating population diversity. In contrast, a smaller value would encourage the exploitation of current solutions and speed up the population convergence. In TMO-MFEA, a larger *rmp* is used for diversity-related variables (DV) to enhance its diversity, while a smaller *rmp* is designed for convergence-related variables (CV) to achieve a better convergence [119,120]. Particularly, *rmp* for CV equals to 0.3, and *rmp* for DV equals to 1, which means a random assortative mating.

An appropriate parameter is essential to the efficiency and effectiveness of MTEC algorithm, and vice-versa. However, the user-defined and fixed parameter in MFEA and its variants is likely to have some distinct disadvantages. Firstly, the *rmp* parameter is manually specified based on the intuition of a decision maker. It is indeed patently clear that such an offline *rmp* assignment scheme is heavily dependent on the existence of prior knowledge about the different optimization tasks. Given the lack of prior knowledge, particularly in general black-box optimization, inappropriate (blind) *rmp* values risks the possibility of harmful inter-task knowledge transfers, thereby leading to significant performance slowdowns [41,79,121]. Secondly, the *rmp* parameter is immutably fixed for all tasks during the optimization process. Similar to biomes symbiosis [122], there are three relationships between source tasks and a target task in an MTO scenario: mutualism, parasitism, and competition. More importantly, the relationship may vary as the population distributions in their corresponding landscapes change. Although this fixed mechanism can make use of the positive knowledge transfer in some very special cases, it may intuitively bring negative effects in general cases [83].

#### 4.4.2. Parameter Adaptation Strategy

If an optimization task is improved more times by the offspring from other tasks, the probability of knowledge transfer should be increased; otherwise, we will decrease this rate [122,123]. Thus, the probability is defined by

$$rmp\_k = \frac{R\_k^o}{R\_k^s + R\_k^o} \tag{21}$$

where *Rsk* and *Rok* are the proportions of times that the current best solution in subpopulation *Pk* is improved by the offspring of the same task and other tasks, respectively. In addition to the transfer rate, the size of the selected candidate solutions also influences the effect of information transfer. An adaptive control mechanism for the size for each task was also devised in [123].

$$|\mathbb{C}\_k| = rmp\_k(|Ofsp| - |Ofsp\_k|) + |Ofsp\_k|\tag{22}$$

In MPEF (multi-population evolution framework), this parameter was adaptively determined based on evolution status [82,83]:

$$rmp\_k = \begin{cases} \min(rmp\_k + c \cdot tsr\_k, 1), & tsr\_k > sr\_k\\ \max(rmp\_k - c \cdot (1 - tsr\_k), 0), & tsr\_k < sr\_k \end{cases} \tag{23}$$

where *srk* is the success rate of subpopulation *Pk*, *tsrk* is the success rate of that offspring generated with the genetic material transfer, and *c* is a constant parameter.

A simple random searching method was introduced to adjust this parameter [94]. The current *rmp* is stored in the candidate list when at least one of *K* best solutions is updated by a better solution. Otherwise, the parameter is adapted as follows:

$$rmp\_k = rmp\_k + \delta \cdot N(0, \ 1) \tag{24}$$

where *δ* is a constant parameter, and *N* (0,1) is a Gaussian noise with zero mean and unit variance.

Based on the saturation point of the knowledge transfer (SPKT), the knowledge transfer control scheme was proposed to control the generation of hybrid-offspring and alleviate the harmful transferred knowledge [99]. Based on the efficiencies of the global search and local search component, Liu et al. [86] proposed an adaptive control strategy, which can determine whether to perform the global search (DE) or the local search (CMA-ES) during the evolution.

Further, Binh et al. [124] proposed a new method for automatically adjusting *rmp* parameter. Specifically, the separate *rmp* value for each task is updated by

$$rmp[i] = \frac{\mathbf{S}\_{\mathbf{r}\_i, NF = 0}}{NP\_i} \tag{25}$$

where *NPi* is number of individuals in the current task, *<sup>S</sup><sup>τ</sup>i*,*NF*=<sup>0</sup> is the set of individuals with skill factor *τi* and belong to the first nondominated front. The idea behind this definition is that, when most of the individuals are in the first nondominated front, the search process may ge<sup>t</sup> stuck in a local nondominated front and then we should increase RMP parameter for the cross-task crossover.

Besides, Zheng et al. [125] defined a novel notion of ability vector to capture the correlations between different tasks and automatically changed the intensity of knowledge transfer across tasks to enhance the performance of MTEC algorithm.

It was very recently reported that an enhanced MFEA called MFEA-II was presented, which enables an online parameter *rmp* estimation scheme in order to theoretically minimize the negative interactions between distinct optimization tasks [41]. Specifically, the extent of transfer parameter matrix is learned and adapted online based on the optimal blending of probabilistic models in a purely data-driven manner. Bali et al. [79] further presented a realization of a cognizant evolutionary multi-task engine. This framework learns inter-task relationships based on overlaps in the probabilistic search distributions derived from data generated during the search course. Recently, it was also used to solve the operation optimization of integrated energy systems [121].

Some concepts and operators of the parameter adaptation strategy utilized in MFEA-II cannot be directly applied to permutation-based discrete optimization environments, such as parent-centric interactions. Osaba et al. [126] entirely reformulated such concepts, making them suitable to deal with discrete optimization problem without losing the

inherent benefits of MFEA-II. Furthermore, dMFEA-II implements a novel and simple strategy for dynamically updating the RMP matrix to the search performance.

#### 4.4.3. Resource Reallocating Strategy

Recently, resource reallocating strategies in MFEA were integrated, which allocate the computational resources according to the complexities of tasks. For example, Wen and Ting [104] proposed an MFEA with resource allocation, named MFEARR. It can determine the occurrence of parting ways during evolution, at which time the effective cross-task knowledge transfer begins to fail. Then, an adaption strategy was proposed, where the transformation frequency is proportional to the probability of positive knowledge transfer. Gong et al. [127] put forward a MTO-DRA algorithm to enable dynamic resources allocation according to task requirements, such that more computing resources are assigned to complex tasks. Motivated by the similar idea that the limited computing resources should be adaptively allocated to different tasks, Yao et al. [128] also proposed dynamic resource allocation strategy. During the evolution of the population, individuals with high scalar fitness will ge<sup>t</sup> more investments or rewards, that is, more computing resources are allocated to them, and the scalar fitness of each individual is measured by a utility and updated periodically.

#### *4.5. Evaluation and Selection Strategy*

General speaking, the complete definition of a universal selection operator is composed of evaluation, comparison, and selection methods. The individual's performance can be evaluated directly or indirectly [51]. As an indirect method, the scalar fitness was originally proposed in MFEA and its variants [18,57]. On the other hand, the fitness value of objective function is a nature and typical direct method [82,83,86,88,122]. Note that scalar fitness and function fitness are equivalence relations in a multi-task scenario [51].

After evaluating all individuals' performances (function fitness or scalar fitness), the next question is the scope or level of comparison objects. In MFEA, the *offspring-pop* (*Rt*) and *current-pop* (*Pt*) were concatenated and then a sufficient number of individuals were selected to yield a new population [18]. This approach can be called population-based (or all-to-all) comparison. As a contrast, individual-based (or one-to-one) comparison was also utilized [61,82–84,88]. Once the offspring individual is generated by intra-population or inter-population reproduction, it is compared with its parent directly and then the better one can remain in the next generation.

For the case of population-based comparison, some alternative strategies were proposed to select the fittest individuals from the joint population. For example, MFEA and its variation follow elitist selection [18], level-based selection [53], and self-adaptive parent selection [129]. Furthermore, it may remove the worse or redundant individuals so as to create more population diversity [61].

The existing MTEC algorithms adopt a fitness-based selection criterion for effectively transferring elite genes across tasks. However, population diversity is necessary when it becomes a bottleneck against the genetic transfer. In [130], Tang et al. proposed a new selection criterion keeping a balance between individual fitness and population diversity as follows:

$$\min\_{i} \{ \mathfrak{a} \cdot p\_{i}.FS + (1 - \mathfrak{a}) \cdot p\_{i}.CD \} \tag{26}$$

where *α* is the balance factor, *FS* is fitness scalar which can adjust factorial cost of individuals evaluated for different tasks to a common scale, and *CD* is crowding distance which can approximately estimate individual diversity.

#### **5. Related Extension Issues of Multi-Task Evolutionary Computation**
